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1.  Introduction 

This  research  program  addresses  an  unmet  need  in  the  training  for  treatment  of  head  trauma,  focusing 
on  ocular,  peri-ocular  and  facial  trauma.  While  improvements  in  armor  systems  have  reduced  injuries  to 
the  trunk  and  extremities,  the  face  and  head  remain  relatively  vulnerable.  Opportunities  for  medics  to 
learn  the  treatment  of  facial  and  head  trauma  are  limited  due  to  the  lack  of  realistic  training  simulators 
for  this  area  of  the  body.  For  military  ophthalmologists,  critical  trauma  skills  such  as  microsurgical 
suturing  are  no  longer  routinely  taught  in  US  residencies.  Consequently,  the  burden  of  developing  and 
maintaining  these  essential  trauma  skills  is  on  the  ophthalmologists  themselves.  We  are  addressing 
these  dual  needs  by  creating  a  new  prototype  training  system  to  simulate  injuries  of  the  orbit  and  peri¬ 
orbital  tissues,  followed  by  the  mandible,  maxilla,  and  upper  airway.  The  system  will  function  as  a 
procedural  training  system  and  will  also  be  designed  for  integration  into  a  mannequin-based  trauma 
simulator  for  advanced,  multiple  trauma  scenarios.  It  is  being  designed  to  incorporate  performance 
measurement  sensors  and  algorithms  with  physical  mannequin  components,  conventional  displays  and 
augmented  presentations  of  teaching  content,  and  provide  meaningful  feedback  to  enhance  learning. 
The  research  is  being  performed  in  three  overlapping  phases:  improvement  of  an  earlier  proof-of- 
principle  eye  trauma  system  for  preliminary  user  testing;  substantial  revision  and  expansion  of  the  eye 
trauma  system  to  form  a  full  face/neck  trauma  system  with  additional  eye  trauma  and  face  trauma 
components;  and  development  and  integration  of  a  mannequin-compatible  version  of  relevant 
components  to  enable  more  comprehensive  trauma  scenarios. 

This  program  was  originally  conceived  as  a  two  year  research  and  development  effort;  a  no  cost 
extension  request  was  submitted  to  enable  completion  of  the  work  which  has  been  subject  to  various 
delays. 

This  report  presents  the  accomplishments  of  the  second  year  of  the  program. 

2.  Body 

2.1.  Project  background:  proof-of-principle  simulator  development 

The  origin  of  this  research  program  is  in  proof-of-principle  system  developed  in  our  laboratory  in 
response  to  training  needs  expressed  by  our  subject  matter  expert.  Dr.  Robert  Mazzoli,  now  of  the 
Vision  Center  of  Excellence.  In  that  earlier  research.  Dr.  Paul  Neumann  and  colleagues  created  a 
physical  silicone  head  form  with  a  replaceable  left  eye  module,  and  coupled  it  to  a  pair  of  magnetic 
position  tracking  sensors  attached  with  Velcro  straps  to  the  surgeon's  index  fingers.  In  performing  lid 
laceration  repair,  the  index  figure  motions  were  measured,  with  performance  statistics  previously 
employed  in  a  laparoscopy  trainer  that  we  developed  Error!  Reference  source  not  found.,  including 
total  exercise  duration,  instrument  path  length,  an  indication  of  motion  smoothness,  and  a  measure  of 
amount  of  hand  reorientation.  The  original  project  included  demonstrations  for  the  subject  matter 
experts  and  others,  but  did  not  extend  to  the  level  of  collecting  expert  and  novice  performance  data  to 
create  a  means  for  distinguishing  the  groups  and  determining  when  a  novice  reached  some  threshold  of 
acceptable  performance. 
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Figure  1.  Proof-of-principle  physical  mannequin,  motion  tracking  system,  sample  results  plot,  circa  2010. 

While  a  strong  beginning,  numerous  desirable  changes  were  identified  by  Dr.  Mazzoli  and  others. 
Anatomically,  the  eye  lid  consisted  of  only  one  layer  of  silicone;  real  lids  have  two  major  layers  of 
differing  biomechanical  character  and  additional  structures  and  landmarks  important  to  correct  surgical 
repair.  A  hollow  eye  globe  was  filled  with  a  simulated  vitreous  humor  substitute,  however  the  cornea 
was  opaque  and  the  anterior  chamber  was  a  solid  structure;  for  realism  and  to  allow  simulated  injury  of 
the  globe,  a  transparent,  thin  cornea  is  necessary,  together  with  an  iris/pupil,  which  should  indicate  light 
response  or  pathology.  The  lid  lacerations  were  manually  created,  with  limitations  on  how  repeatable 
the  simulated  injury  could  be  generated;  a  training  system  with  standardized,  replicable  scenarios  would 
be  more  useful  and  reliable  as  a  measurement/testing  system.  The  tracking  sensors  were  attached  to 
the  user's  fingers,  providing  at  best  a  poor  approximation  of  the  motions  of  the  instrument  tips;  revision 
of  this  and  the  expansion  of  the  instrument  suite  were  requested.  Lastly,  while  the  laparoscopy 
simulator-based  scoring  system  used  has  been  shown  to  be  able  to  distinguish  expert  from  novice 
laparoscopic  surgeons  (on  a  suitable  simulator),  it  is  a  post-performance  score  and  cannot  easily  be  used 
to  make  detailed  recommendations  for  improvement  -  a  more  detailed  evaluation  and  feedback  system 
is  desirable. 

In  addition  to  addressing  these  system  improvements,  the  original  vision  was  to  create  a  larger  suite  of 
scenarios,  to  be  supported  by  a  series  of  replaceable  eye  trauma  modules:  globe  laceration  (scleral 
and/or  corneal),  more  complex  lid  lacerations,  and  retrobulbar  hemorrhage. 


The  current  research  also  extends  the  simulator  to  create  a  system  that  supports  training  of  first 
responders  in  management  of  eye  trauma  as  well.  As  ocular  injuries  are  typically  addressed  surgically 
only  after  life  threatening  conditions  are  managed,  it  is  essential  that  first  responders  properly  protect 
the  injured  eye  prior  to  transport,  avoid  causing  additional  damage  (typically  by  applying  inappropriate 
pressure),  and  in  the  case  of  retrobulbar  hemorrhage,  quickly  identify  the  condition  and  take 
appropriate  steps  to  relieve  pressure  on  the  optic  nerve  to  avoid  permanent  loss  of  vision.  For  this 
reason,  developing  capabilities  to  present  eye  injuries  to  medics  and  detect  interventions  was  deemed 
to  be  important  for  ongoing  development.  Because  eye  trauma  is  often  accompanied  by  face  trauma, 
including  additional  facial  trauma  elements  and  airway  management  features  were  also  considered. 
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together  with  integration  of  a  revised/improved  simulator  with  a  full-body  mannequin-based  simulator 
for  combined  head/body  trauma  capabilities. 

These  needs  lead  to  the  current  research  effort,  which  seeks  to  address  them  in  three  overlapping 
phases: 


•  Revision  and  evaluation  of  the  proof-of-principle  eye  trauma  system,  including  user  testing 

•  Expansion  of  the  eye  trauma  system  to  include  additional  ocular  elements,  facial  trauma 
components  and  upper  airway  management  elements 

•  Integration  of  the  procedural  trainer  components  into  a  mannequin  system 

The  second  year  progress  towards  these  areas,  described  by  the  various  milestones  related  to  these 
phases,  is  detailed  in  the  sections  below. 


2.2.  Quarterly  Report  Administrative  Details 

As  this  report  includes  all  required  elements  of  the  Q8  quarterly  report,  items  1-10  are  included  here. 
Items  11  and  12  (Scientific  Progress,  and  Plans/Milestones  statement)  are  reported  in  the  remainder  of 
the  body  (section  2.3  and  its  subsections)  and  the  key  research  accomplishments  (section  3)  and 
reportable  outcomes  (section  4),  corresponding  with  the  Annual  Report  format. 

1.  Contract  No.  _  W81XWH-11-C-0095 _ 

2.  Report  Date  8/10/2013 

3.  Reporting  period  from  4/11/2013 _ to  7/10/2013 _ 

4.  PI  _Mark  P.  Ottensmeyer,  Ph.D. _ 

5.  Telephone  No.  617-768-8783 

6.  Institution  Massachusetts  General  Hospital _ 

7.  Project  Title_Development  of  an  Ocular  and  Craniofacial  Trauma  Treatment  Training  System _ 

8.  Current  staff,  with  percent  effort  of  each  on  project: 


Mark  P.  Ottensmeyer,  Ph.D. 

P.l. 

89% 

Steven  L.  Dawson,  M.D. 

co-l,SME 

20% 

Gianluca  De  Novi,  Ph.D. 

Computer  Scientist,  Post-Doc 

100% 

Gregory  J.  Loan 

Clinical  Engineer,  Fabrication  Lead 

90% 

Maria  J.  Troulis,  D.D.S. 

SME 

10%^ 

1.  Dr.  Troulis  will  be  continuing  to  serve  as  an  SME  during  the  period  of  the  requested  no-cost 
extension,  with  no  further  salary  component  requested. 


9.  Contract  expenditures  to  date  (as  applicable): 

This  Qtr  / 

Personnel:  $92,131.66  / 

Fringe  Benefits:  35,173.55  / 

Travel:  5,253.32  / 

Equipment:  0.00  / 


Cumulative 

$816,841.77 

292,586.08 

22,410.73 

139,858.50 
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Non-capital  equipment: 
Supplies: 

Consultants: 

Other: 


11.98  / 

3,227.08  / 

0.00  / 
10,946.47  / 


27,702.70 

66,350.28 

15,346.40 

36,430.90 


Subtotal: 
Indirect  Costs: 
Fee: 

Total: 


146,744.06  / 

108,590.68  / 

0.00  / 
255,334.74  / 


$1,449,307.84 

490,832.80 

0.00 

$2,427,748.03 


10.  Comments  on  administrative  and  logistical  matters. 

A  summary  of  all  of  the  quarterly  report  logistics/administration  sections  is  summarized  here,  and  the 
new  elements  that  were  relevant  to  Q8  are  described,  with  bold  indicating  the  quarter. 

From  an  overall  perspective,  as  has  been  previously  reported,  personnel  availability  limitations  have 
lead  to  lags  behind  the  originally  anticipated  progress  schedule.  While  performance  is  consistent  with 
expenditure  levels,  completion  of  the  program's  planned  deliverables  will  require  an  extension  of  the 
performance  period  of  this  effort.  Pursuant  to  this,  in  Q7,  a  no-cost  extension  request  was  submitted 
(29  March  2013),  to  extend  the  period  of  performance  by  a  total  of  12  months,  to  July  10,  2014.  This 
extension  anticipates  that  the  research  and  development  components  of  the  work  will  be  completed 
early  in  2013.  Following  this,  we  expect  a  reduction  of  effort  level  and  salary  expenditures  for  the 
remainder  of  the  period,  as  remaining  work  is  expected  to  focus  mainly  on  publications,  conference 
presentations,  and  potentially  a  third  user  trial/demo  at  the  2014  USUHS  eye  trauma  course  in  May 
2014. 

At  the  close  of  Q8,  the  response  to  the  NCE  request  remained  pending. 

Personnel  updates: 

Over  the  course  of  Year  2,  we  have  seen  turnover  in  some  of  our  permanent  staff,  primarily  due  to 
outside  opportunities,  counteracted  in  part  by  the  recruitment  of  new  personnel,  and  we  have 
benefited  from  the  contributions  of  an  ongoing  series  of  graduate  student  interns. 

In  Q5,  Mr  Raj  Shah,  clinical  engineer,  who  had  been  working  part  time  between  our  group  and  the  Voipe 
Center  of  the  Department  of  Transport,  took  on  full  time  status  with  the  DOT.  In  the  same  quarter,  Mr. 
Simone  Zivieri,  mechanical  engineering  intern,  began  work  to  learn  design  and  fabrication  techniques 
with  application  to  developing  the  eye  motion  and  blinking  mechanisms  for  the  system. 

During  Q6,  Dr.  Bummo  Ahn,  our  post-doctoral  research  fellow  in  mechanical  engineering,  completed  his 
training  with  us  and  returned  to  his  home  country  to  a  position  in  industry.  His  work  on  the  neck 
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mechanism  and  various  sensing  systems  was  picked  up  by  Dr.  Ottensmeyer.  In  the  same  quarter,  Mr. 
John  Moore  took  a  permanent  position  as  a  director  of  product  design  for  a  commercial  entity.  Mr. 
Ryan  Bardsley  took  on  the  anatomical  modeling  and  fabrication  duties  of  Mr.  Moore.  Ms.  Martina  Festi, 
master's  student  intern  in  computer  science  began  her  work  with  us,  learning  coding  techniques  in  C# 
and  C++  as  applied  to  communications  between  our  simulation  system  and  the  operation  of  the  Laerdal 
SimMan  Essential  mannequin,  assembly  code  to  update  the  data  acquisition  system  for  the  stand-alone 
simulator  (developed  by  year  1  intern,  Evgheni  Munteanu),  and  basic  statistical  tools. 

In  Q7,  we  completed  the  hiring  process  for  Mr.  Gregory  Loan,  with  experience  in  molding  and  model 
making,  as  well  as  interfacing  and  control  of  robotic  systems  using  microcontrollers  similar  to  the  ones 
we  are  using.  In  addition,  he  has  brought  the  concepts  and  some  of  the  techniques  of  the  Scrum/Agile 
software  development  methods  to  our  overall  project  management  toolset.  Also  in  this  period,  our 
post-doc  Dr.  Gianluca  De  Novi  was  recommended  for  and  received  promotion  to  full  time  Assistant  in 
Research/Instructor  status.  Intern  Zivieri  completed  his  work  term  with  us  successfully  and  returned  to 
his  studies  to  complete  his  degree  requirements,  graduating  cum  laude.  Dr.  Ottensmeyer  and  Mr. 
Bardsley  obtained  training  in  the  use  of  laser  cutting  at  a  local  service  shop,  to  expand  the  range  of 
fabrication  capabilities  within  the  group. 

Q8  saw  the  departure  of  one  of  the  group's  longest  serving  members,  Mr.  Ryan  Bardsley,  who  left  to 
take  a  position  with  a  pharmaceutical  firm,  who  had  been  working  with  us  part-time  since  the  outset  of 
the  program.  At  the  close  of  the  quarter,  we  have  not  yet  identified  a  replacement  for  him.  A  new 
mechanical  engineering  intern,  Mr.  Federico  Pifferi,  began  his  work  term  with  us,  focusing  on 
development  of  the  neck  mechanisms,  in  support  of  his  exposure  to  CAD,  rapid  prototyping  and 
fabrication  techniques,  and  developing  control  code  for  neck  motion. 

Review  of  significant  non-personnel  expenses  during  Year  2 

Equipment/Supplies: 

During  Q6,  we  acquired  a  Laerdal  SimMan  Essential  mannequin  system  (equipment),  to  which  we  will 
be  attaching  the  mannequin-compatible  version  of  the  trauma  head/neck.  In  addition,  we  extended  the 
1-year  license  to  a  perpetual  license  for  the  Materialise  Mimics/3-Matic  software  tools  for  segmentation 
of  CT  anatomical  scans  and  conversion  of  that  data  into  computer  aided  design  formats.  The  annual 
maintenance  fee  for  the  Objet  Eden  260V  3D  printing  system  was  incurred,  as  was  the  fee  for  our 
Solidworks  CAD  software.  In  addition,  after  a  year  of  good  service,  one  of  the  print  heads  on  the  Objet 
system  had  to  be  replaced  to  ensure  ongoing  good  quality  parts  fabrication. 

During  Q7,  Dr.  Ottensmeyer's  main  laptop  computer  failed  permanently,  requiring  replacement,  and  the 
annual  maintenance  fee  for  the  Stratasys  FDM  3D  printing  system  was  incurred. 

Consultants: 
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In  Q8,  we  again  turned  to  our  consultant  Mr.  Daniel  Langston,  who  has  previously  fabricated  copies  of 
our  trauma  modules,  to  create  the  modules  that  were  used  by  trainees  and  staff  at  the  2013  USUHS 
Ocular  Trauma  course  user  testing  exercise  of  our  simulator. 

Travel: 

During  Q5,  Dr.  Ottensmeyer,  and  now-former  team  members  Mr.  Bardsley  and  Mr.  Shah  attended  the 
2012  MHSRS/ATACCC  meeting  in  Ft.  Lauderdale,  FL  (13-16  August  2012),  to  present  a  research  poster  on 
the  project,  perform  demonstrations  to  TATRC  personnel  and  our  SME  Dr.  Mazzoli  and  colleagues  of  his. 

During  Q6,  Dr.  Ottensmeyer,  Dr.  De  Novi  and  Mr.  Bardsley  attended,  gave  an  oral  presentation  and 
demonstrated  the  simulator  system  at  the  TATRC  booth  at  the  2013  MMVR  conference  (20-23  February 
2013). 

In  Q8,  Dr.  Ottensmeyer,  Dr.  De  Novi  and  Mr.  Loan  traveled  to  USUHS  to  conduct  the  second  user  testing 
exercise  at  the  2013  Ocular  Trauma  course  (20-23  May  2013),  and  Dr.  Ottensmeyer  traveled  to  TATRC  to 
present  a  progress  report  before  the  JPCl  committee  members  (19  June  2013). 

IRB/Human  use  protocol  status: 

During  Q6,  the  Biosafety  committee  protocol  covering  the  use  of  the  human  tissue  acquired  for  our 
early  work  towards  obtaining  anatomical  models  from  CT  and  MR  scanning  and  molding  methods  was 
concluded.  As  we  are  using  the  high  resolution  CT  imagery  from  our  earlier  neuroendovascular 
simulator  project,  we  did  not  renew  the  biosafety  protocol.  In  addition,  in  the  context  of  developing 
additional  research  topics,  we  made  contact  with  Dr.  Mannudeep  Kaira  of  MGH  Imaging,  who  has  a 
project  to  develop  a  virtual  reality  cadaver/anatomy  visualization  system.  He  has  acquired  high 
resolution  data  sets  of  over  100  deceased  patients.  We  requested  permission  to  examine  these  data 
sets,  and  were  added  to  his  protocol  towards  the  end  of  the  quarter.  Other  element  of  the  effort  have 
had  higher  priority  than  this  examination,  however  we  retain  the  opportunity  to  examine  these  data 
going  forward. 

In  Q7,  pursuant  to  a  return  to  the  USUHS  ocular  trauma  course,  we  submitted  the  continuing  review 
documentation  to  our  IRB  and  submitted  an  amendment  to  cover  the  extended  period,  additional 
cohort  of  participants,  change  in  the  context  of  the  human  study  (simulator  session  vs.  independent 
event),  and  a  request  to  collect  additional  demographic  data  regarding  prior  medical  and  simulator 
training.  Our  IRB  issued  its  approval  without  request  for  any  changes.  We  forwarded  these  documents 
to  the  ORP  HRPO  for  their  review,  which  also  issued  its  approval  for  the  ongoing  work.  The  vendor 
agreement  that  was  prepared  between  MGH  and  USUHS  to  permit  bringing  our  simulator  to  the  course 
was  also  completed  during  this  quarter. 

In  Q8,  following  the  second  user  trial,  we  submitted  a  minor  amendment  to  our  IRB  to  confirm  that  we 
may  merge  certain  demographic  details  collected  as  part  of  the  user  testing,  with  the  responses 
obtained  from  a  user  feedback  survey  that  was  developed  to  aid  the  Ocular  Trauma  course  organizers  in 
evaluating  the  use  of  simulators  for  future  programs;  responses  to  the  survey  could  therefore  be 
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stratified  between  responses  from  residents  and  those  from  ophthalmologists,  rather  than  lumping  all 
of  those  responses  together.  The  HRPO  expressed  the  opinion  that  they  did  not  require  review  of  minor 
amendments  such  as  this  one,  however  minor  amendments  will  still  be  sent  to  TATRC  regulatory 
personnel  for  their  review  and  awareness.  Also  related  to  the  USUHS  user  testing,  Mr.  Loan  completed 
his  CITI  training  to  permit  him  to  participate  in  human  studies  work,  and  an  amendment  was  approved 
to  add  him  to  the  study  staff.  Another  minor  amendment  was  approved  by  our  IRB  to  permit  statistical 
analysis  to  be  performed  by  Dr.  Paul  Latkany  at  the  Vision  Center  for  Excellence  and  his  colleagues,  who 
have  background  in  this  area  beyond  the  capabilities  of  the  Simulation  Group  team.  Lastly,  with  TATRC 
regulatory  personnel,  we  confirmed  that  the  use  of  the  deidentified,  anonymized  CT  data  set  being  used 
to  derive  the  anatomy  for  the  revised  version  of  the  head/neck  anatomy  (to  replace  the  original  life-cast 
silicone  head),  is  considered  no-human-use. 

SME  and  other  expert  interactions: 

We  have  continued  to  maintain  ongoing  contact  with  our  primary  SMEs,  Dr.  Robert  Mazzoli  (Vision 
Center  of  Excellence),  Dr.  Maria  Troulis  (Massachusetts  General  Hospital),  Dr.  Suzanne  Freitag 
(Massachusetts  Eye  and  Ear  Infirmary),  and  re-establish  and  cultivate  new  contacts  to  provide  expertise, 
feedback  and  additional  motivation  for  the  work. 

We  have  had  in-person  discussions  with  Dr.  Mazzoli  at  events  including  the  Military  Vision  Research 
Symposium  (18-20  Sept  2012)  and  workshop  on  Developing  a  Roadmap  for  Simulation  in  Eye  Care  (17- 
19  Oct  2012),  as  well  as  having  him  review  the  status  of  the  system  at  the  2013  USUHS  Ocular  Trauma 
course  (20-23  May  2013)  and  maintaining  periodic  e-mail  contact.  We  have  approximately  quarterly 
meetings  with  Dr.  Freitag,  including  discussions  to  develop  step-by-step  outlines  of  surgical  procedures 
and  reviews  of  the  simulator  development  and  anatomical  material  behavior,  including  a  visit  at  the  end 
of  Q8  to  aid  in  calibration  of  the  force  sensing  systems  of  the  surgical  instruments.  We  continue  to 
discuss  plans  for  facial  fracture  scenarios  with  Dr.  Troulis,  and  will  increase  our  contact  with  her  during 
the  NCE  period  as  we  shift  focus  to  the  facial  components  of  the  simulator.  Her  current 
recommendation  is  to  develop  a  bilateral  mandibular  fracture  module,  which  would  contribute  to 
collapse  of  the  oral  cavity,  leading  to  the  requirement  to  perform  a  surgical  airway. 

Additional  interactions  over  year  2  have  included: 

•  Dr.  Paul  Latkany,  Vision  Center  of  Excellence,  who  has  interests  in  developing  an  eye  trauma 
registry,  and  also  the  statistical  analysis  of  data  related  to  training  for  eye  surgery.  At  the  USUHS 
Ocular  Trauma  course,  we  met  with  colleagues  of  his  with  statistical  analysis  backgrounds,  who 
may  be  able  to  provide  advice  in  development  tools  for  validation  of  the  gesture  detection 
algorithms  and  of  scoring  methods  for  distinguishing  experts  from  novice  users  of  the  simulator. 

•  COL  Robert  Hale,  ISR,  in  Q8,  as  recommended  by  TATRC  personnel  following  the  JPCl 
presentation  in  June  2013,  to  follow  up  on  our  early  discussions  regarding  the  training  needs  for 
facial  trauma.  In  this  conversation,  it  was  reiterated  that  there  is  a  strong  need  for  a  realistic 
simulation  of  facial  trauma,  including  elements  such  as  VR  atlases  of  trauma  and  physical 
simulators  that  could  enable  the  teaching  of  wiring  techniques,  and  that  physicians  outside  of 
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level  1  trauma  centers  need  simulators  as  they  have  limited  exposure  to  facial  trauma 
otherwise. 

•  COL  Patrick  Storms,  USAF,  who  was  developing  a  training  program  for  physicians  in  the  reserves, 
scheduled  for  summer  2013  and  inquired  regarding  whether  our  simulator  could  be  used, 
however  that  event  was  scaled  back  due  to  funding  limitations,  so  we  did  not  participate 

•  CDR  Chris  Cornelissen,  USN,  at  NAVMEDWEST,  with  whom  we  discussed  his  training  programs 
involving  cadaver  use.  While  we  could  not  coordinate  a  visit  by  him  to  view  our  system  at  the 
MMVR  2013  TATRC  booth,  we  will  attempt  to  reestablish  that  contact  for  a  demo  at 
MMVR2014,  to  be  held  near  Los  Angeles. 

•  Dr.  Mannudeep  Kaira,  Radiology,  MGH,  who  has  established  a  research  program  to  study  high 
resolution  CT  image  sets  of  cadavers,  with  over  100  subjects  in  his  archives.  We  have  discussed 
exploration  of  his  data  for  potential  guidance  on  developing  the  facial  fracture  and  airway 
components,  as  well  as  development  of  a  spin-off  technology  from  the  gesture  recognition  work 
in  the  form  of  a  virtual  autopsy  table  with  a  natural  motion  user  interface.  A  proposal  for  a 
small  grant  was  submitted  to  support  this  project,  however  in  a  large  field  of  applicants,  it  was 
not  selected  for  funding. 

•  Dr.  Diane  Wright  and  Dr.  Timothy  Toth,  MGH  Fertility  center,  who  have  expressed  interest  in  the 
concept  of  a  VR  simulator,  based  on  our  augmented  reality  surgical  microscope  work,  to  teach 
their  staff  and  those  of  other  centers  techniques  for  micromanipulation  of  egg  and  sperm  cells, 
without  need  for  tissue-based  training  and  use  of  the  expensive,  real  manipulator.  In  Q7,  we 
visited  their  facility  to  observe  the  procedure,  and  in  Q8  followed  up  with  a  demonstration  of 
our  system  and  capabilities. 

•  Neil  Schell,  Director,  Business  Development,  Polhemus,  who  we  met  at  MMVR  2013,  and  who 
provided  us  with  an  evaluation  loan  of  their  sensor  technology  so  that  we  could  compare  it  with 
the  existing  Ascension  hardware  in  Q8.  We  found  their  sensor  cable  flexibility  to  be  superior, 
however  the  sensors  themselves  were  very  large  compared  with  the  smaller  surgical 
instruments)  and  there  appeared  to  be  no  price  advantage.  In  the  course  of  the  testing,  we  did 
uncover  that  the  Ascension  sensors  should  not  necessarily  be  used  by  people  with  pacemakers 
and  similar  life-saving  devices;  we  have  amended  the  IRB  protocol  for  user  studies  to  exclude 
people  in  this  class. 

•  At  MMVR2013,  we  met  with  Ken  Gracey,  president  of  Parallax,  Inc.,  in  reference  to  the 
MicroMedic  program  sponsored  by  TATRC.  We  acquired  one  of  the  kits,  and  expect  to  make  use 
of  the  technology  in  the  following  year.  Also  at  MMVR2013,  we  met  with  COL  Lenhart  and  Mr. 
Magee  to  gage  interest  in  a  number  of  potential  project  ideas,  including  the  AR  autopsy  table, 
and  investigating  whether  physical  therapy  might  benefit  from  the  development  of  a  training 
mannequin  system. 

•  Dr.  Pamela  Andreatta,  who  Dr.  Ottensmeyer  met  at  the  JPCla  in-progress  review  meeting  on 
June  19,  and  her  colleagues  who  had  interest  in  the  simulator  for  use  in  emergency  medicine 
training.  Conversations  have  been  scheduled  for  12  July  2013. 

•  Beth  Pettit  and  Jack  Norfleet  of  ARL,  following  on  from  inquiries  at  the  JPCla  meeting  regarding 
whether  the  system  could  be  attached  to  other  mannequin  systems.  This  conversation  also  lead 
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to  an  invitation  to  ARL  to  participate  in  a  Massachusetts  General  Hospital  proposal  in  response 
to  the  USAMRMC  solicitation  for  the  Advanced  Modular  Mannequin  project. 

Funding  Proposals: 

During  this  year,  we  have  been  working  to  prepare  two  proposals  related  to  this  program,  namely,  a 
response  to  the  solicitation  for  the  Advance  Modular  Manikin  program  (W81XWH-13-R-0032),  and  a 
response  to  a  call  for  proposals  issued  by  CIMIT  towards  a  combined  program  addressing  the  Joint 
Warfighter  Medical  Research  Program.  As  the  year  ended,  we  had  initiated  discussions  towards  a 
sponsored  research  agreement  to  develop  a  modular  ear  anatomy  simulator  that  will  adopt  a  number  of 
the  features  of  the  eye  trauma  sim. 

For  the  AMM  program,  we  are  proposing  to  adapt  the  mannequin-compatible  version  of  the  eye  trauma 
head/neck  system  for  use  as  one  of  the  peripheral  units  that  would  be  compatible  with  and 
demonstrated  as  a  part  of  a  new  modular  manikin.  The  concept  of  adapting  the  head-neck  component 
for  use  on  additional  mannequin  systems  was  brought  following  the  presentation  given  at  the  JPCla  In- 
Progress  Review  on  19  June  2013,  at  which  point  we  confirmed  that  adaptation  of  the  system  to 
mannequins  other  than  the  Laerdal  SimMan  Essential,  which  we  are  using  for  our  mannequin  platform, 
was  feasible. 

For  the  JWMRP  proposal,  we  are  preparing  a  submission  with  three  elements  of  work,  namely  creation 
of  a  surgical,  mid-face  fracture  module,  creation  of  an  enucleation  module,  and  adaptation  of  the  head- 
neck  system  for  compatibility  with  another  modular  mannequin  system  currently  under  development  at 
the  Massachusetts  General  Hospital  Learning  Laboratory.  This  latter  mannequin  system  is  aimed  at  a 
low-cost  market,  without  many  of  the  advanced  features  called  for  in  the  AMM  solicitation,  however  the 
architecture  under  development  would  be  compatible  with  the  creation  of  physical,  power  and  data 
adapters  to  allow  attachment  of  our  head. 

The  ear  anatomy  simulator  is  in  response  to  inquiries  from  a  local  company  developing  a  3D  scanning 
technology  for  ear  canals.  Our  anatomical  modeling  work  and  data  acquisition  tools  for  the  eye  trauma 
sim  are  expected  to  form  elements  of  a  new  system.  The  sponsored  research  agreement  is  expected  to 
support  development  of  a  prototype  that  will  be  replicated  by  a  third  party  vendor. 

Further  details  on  these  proposals  will  be  reported  in  future  reports. 

2.3.  Research  Accomplishments  by  Milestone 

For  each  of  the  individual  milestone  tasks,  a  review  of  the  work  over  the  relevant  periods  of  the  program 
(previously  described  in  quarterly  reports  5  -  7  and  monthly  technical  reports  6  -  9)  is  presented.  Where 
new  work  has  been  performed,  it  follows  the  review  and  its  significance  relative  to  the  earlier  work  is 
presented. 

The  ''Month  #"  reference  follows  the  language  of  the  contract,  although  progress  proceeds  in  parallel 
across  multiple  milestones  and  completion  of  the  milestones  varies  from  that  calendar. 
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In  the  proposal,  certain  separate  milestones  were  listed  for  completion  in  the  same  month,  but  were 
combined  in  the  contract  language.  In  this  overview,  such  items  are  separated  as  parts  a,  b,  etc.  where 
relevant. 

2.3.1.  “Month  1”  -  Review  of  craniofacial  trauma  training  needs  with  Subject  Matter 
Experts  (SMEs),  creation  of  specifications,  limitations,  detailed  analysis  for  trauma 
cases. 

This  milestone,  relevant  to  the  Phase  I  work,  was  completed  in  year  1.  As  the  Phase  II  and  Phase  III 
work  continues,  ongoing  feedback  from  SMEs  and  user  testing  will  influence  ongoing  design  decisions, 
as  in  milestones  2.3.7  (completed)  and  2.3.13  (pending  for  year  3).  As  a  whole,  no  changes  have  been 
made  to  the  plans  for  the  types  of  injuries  to  be  simulated  by  the  system.  Corresponding  with 
preliminary  discussions  that  lead  to  the  original  project  scope,  the  scope  of  injury  scenarios  that  the 
system  should  address  include: 

For  surgical  scenarios: 

•  Full  thickness  lid  lacerations 

•  Lid  laceration  with  canalicular  damage 

•  Intact  globe  with  retrobulbar  hemorrhage 

•  Globe  laceration  (scleral  and  corneal) 

•  Combinations  of  the  above 

For  first  responders: 

•  Protection  of  the  eye  using  an  eye  shield  and  stabilization  of  penetrating  objects 

•  Intact  globe  with  retrobulbar  hemorrhage 

•  Removal  of  non-penetrating  foreign  bodies 

•  Facial  hemorrhage  control 

•  Management  of  airway  with  severe  damage  to  the  lower  face 

•  Surgical  airway 

Retrobulbar  hemorrhage  appears  in  both  lists  as  this  is  a  condition  that  requires  recognition  and  urgent 
response  whether  in  the  field  or  at  a  medical  facility,  namely  lateral  canthotomy  and  cantholysis,  is 
included  both  in  surgical  training  and  also  in  the  SOF  Medical  Handbook. 

While  facial  fractures  should  be  included  as  part  of  the  damage  calling  for  hemorrhage  control  and 
management  of  complicated  airway,  surgical  repair  and/or  stabilization  of  the  bone  fragments  is  outside 
of  the  scope  of  this  simulation  system  program.  Similarly,  enucleation  has  been  recommended  as  a 
valuable  scenario.  In  Q8,  as  was  described  earlier  in  this  report,  a  proposal  for  funding  to  create 
modules  for  both  of  these  scenarios  is  in  preparation,  as  part  of  a  call-for-proposals  issued  by  the  CIMIT 
organization,  towards  programs  supporting  the  Joint  Warfighter  Medical  Research  Program. 
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2.3.2.  “Month  2”- 


2. 3. 2.  a.  Review  of  treatment  modes  and  potential  error  modes  to  be  detected 

This  milestone,  relevant  to  the  Phase  I  work,  was  completed  in  Year  1.  As  the  Phase  II  work  continues, 
ongoing  feedback  from  SMEs  and  user  testing  will  influence  ongoing  design  decisions.  These  are 
discussed  further  in  relation  to  reporting  on  the  results  from  the  second  USUHS  Ocular  Trauma  Course 
user  testing  event. 

2.3.2. b.  Implement  improvements  for  upgraded  proof-of-concept  stand-alone  eye 

trauma  simulator  based  on  feedback  from  preliminary  user  testing  by  CIMIT 
project  advisors. 

This  milestone,  relevant  to  the  Phase  I  work,  was  completed  in  Year  1.  The  improvements  to  the 
system  have  incorporated  the  feedback  and  lead  to  the  system  that  was  used  at  the  first  USUHS  Ocular 
Trauma  Course  user  trials,  22-24  May  2012.  Improvements  beyond  these  fall  under  revisions  based  on 
experience  at  USUHS  and  into  the  development  of  the  Phase  II  system. 

2.3.3.  “Month  4”- 

2. 3. 3.  a.  Perform  user  evaluation  at  USUHS  TriService  Ophthalmology  courses. 
The  user  evaluation  element  of  this  milestone  was  completed  in  Year  1. 

The  data  collected  during  this  event  continues  to  be  analyzed  during  Year  2.  During  Q5,  we  reviewed 
the  USUHS  results  with  our  SME,  Dr.  Freitag,  to  identify  features  of  user  performance  that  would 
warrant  feedback  to  the  user.  We  also  created  a  photographic  record  of  the  sutured  modules  that  could 
be  used  to  evaluate  final  performance,  using  expert  review.  This  effort  has  served  as  inspiration  for  a 
more  comprehensive  set  of  photographs  to  be  created  from  the  second  USUHS  trial  modules,  including 
additional  view  angles,  as  recommended  by  Dr.  Mazzoli.  This  is  anticipated  for  Q9  or  QIO. 

A  second  round  of  user  testing  at  the  2013  USUHS  Ocular  Trauma  course  is  described  below  in  section 
2.3.7. 
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Figure  2:  Alignment  rig  and  light  source  for  photographing  sutured  portals.  Examples  of  different  suture  types,  all 
addressing  the  same  injury. 

2.3.3. b.  Locate  and  coordinate  commercial  partner  to  manufacture  first  version  of 
the  stand-alone  eye  trauma  simulator. 

Work  to  investigate  commercialization  is  ongoing. 

During  Q6,  we  established  a  confidentiality  agreement  with  Laeral  and  began  to  hold  technical 
discussions  regarding  how  our  system  can  be  combined  with  the  SimMan  Essential  mannequin.  The 
SimMan  Essential  was  chosen  as  a  platform  for  use  in  this  project  as  it  is  representative  of  relatively  high 
fidelity  mannequin  systems  (i.e.  computer  controlled  physiology,  internal  breathing,  physiological 
systems)  that  would  be  able  to  support  the  electronic  and  fluidic  systems  of  our  system.  The  contact 
with  Laerdal  has  also  supported  our  work  towards  integrating  our  software  systems  with  the  Laerdal 
SDK  used  to  communicate  with  and  control  the  Laerdal  mannequin. 

In  Q7,  following  introductions  made  by  TATRC  at  MMVR  2013,  we  established  a  confidentiality 
agreement  with  Polhemus,  a  provider  of  magnetic  position  tracking  sensors.  This  allowed  us  to  evaluate 
the  Polhemus  sensor  set  as  an  alternative  to  the  Ascension  tracking  system  currently  in  use  in  the  stand¬ 
alone  system  for  measurement  of  instrument  positions.  Our  findings  were  that  the  larger  size  of  the 
Polhemus  sensors  and  the  higher  cost  did  not  justify  switching  to  the  new  system.  One  desirable 
feature  is  the  highly  flexible  cabling  system  that  they  use,  which  would  help  to  avoid  cable 
twisting/tangling  that  we  observe  in  some  users  of  the  existing  system. 

In  Q8,  we  held  discussions  with  Ascension  Technologies  regarding  modifications  of  their  sensors  systems 
to  improve  their  sensors.  Particular  areas  for  enhancement  include  more  flexible  cables  and  integration 
of  additional  signal  wires  into  the  cable,  supporting  the  strain  gage  and  identification  circuits  we 
developed  for  the  instruments.  These  are  features  that  could  be  custom  developed  for  a  future. 


16 


commercial  version  of  the  system,  however  our  immediate  research  and  development  needs  do  not 
justify  the  non-recurring  engineering  costs  that  would  be  involved  in  custom  sensors  at  this  time. 

Also  in  Q8  we  were  approached  by  a  local  start-up  company,  Lantos  Technologies,  who  are 
commercializing  3D  scanning  technology  developed  at  MIT  for  use  in  ear  canal  measurement.  They 
became  aware  of  the  quality  of  the  head  form  developed  for  our  simulator  and  inquired  whether  the 
technology  could  be  adapted  for  use  to  train  otolaryngologists  on  their  device.  We  are  drafting  a  new 
sponsored  research  agreement  towards  development  of  a  commercial  prototype  that  would  be 
replicated  to  at  least  the  level  of  dozens  of  units.  This  system  would  be  a  commercial  application  of  the 
data  acquisition  and  motion/gesture  evaluation  methods  developed  through  this  current  program. 

2.3.4.  “Month  6”  -  Revise  design  and  fabricate  improvements  based  upon  USUHS 
Spring  evaluation. 

This  milestone  is  in  progress.  Based  on  lessons  learned  and  reported  in  the  Year  1  annual  report,  the 
design  of  the  stand-alone  system  has  undergone  substantial  improvement  over  Year  2. 

This  section  will  review  the  progress  in  the  design  of  the  simulator  structure  and  the  instruments.  The 
anatomical  elements  of  the  head  and  eye  trauma  modules  will  be  discussed  in  later  sections. 


2. 3. 4. a.  Stand-alone  simulator  structure  finalization 

At  the  close  of  Q5,  we  presented  a  series  of  concept  drawings  that  began  to  address  limitations  of  the 
system  used  at  the  first  user  testing  exercise. 


Figure  3:  Design  concepts  for  revised  stand-alone  system.  Preliminary  concept  for  blocking  out  system 
components;  early  revision  for  instrument  tray  and  integrated  overhead  lighting  system  by  Mr.  lorino; 
views  of  more  advanced  concepts  including  mobile  arm  to  support  microscope  head,  also  lorino. 

Elements  of  these  designs  were  adopted  through  further  concept  development  that  lead  to  a  new 
instrument  rack  system  shown  at  MMVR2013,  and  an  entirely  revised  system  that  was  used  at  the  2013 
USUHS  Ocular  Trauma  course.  Evolution  of  the  design  concepts  was  reported  in  Q7  and  is  summarized 
below.  These  designs  integrate  new  means  for  mounting  the  surgical  instruments,  optimize  the 
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positioning  of  the  magnetic  and  optical  tracking  systems,  optimize  the  display  field  of  the  projector 
system,  and  notably  support  ease  of  assembly/packing/shipping. 


Figure  4:  Evolution  of  stand-alone  simulator:  modification  of  original  proof-of-principle  system  integrating 
overhead  display  and  lighting  capability;  addition  of  broadened  suite  of  instruments,  instrument  rack  and  early 
augmented  reality  microscope;  revision  prior  to  MMVR  2013  including  suture/instrument  rack  fabricated  using 
laser-cut  panel  construction  and  the  more  robust,  user  friendly  instrument  connectors 


Figure  5:  Evolution  of  design  concepts  for  standalone  system,  as  described  in  Q7  report 


18 


During  Q8,  the  design  shown  in  schematic  form  above  was  reduced  to  practice  and  fabricated  in  time 
for  the  2013  USUHS  Ocular  Trauma  course.  With  minor  ongoing  revisions,  we  expect  that  this  represents 
the  essentially  final  version  of  the  structural  frame  and  main  component  placement  of  the  stand-alone 
simulator.  The  head-form  and  trauma  modules  will  continue  to  be  developed  in  tandem  with 
development  of  the  mannequin-mounted  version,  with  the  new  head-form  being  a  drop-in  replacement 
for  the  version  shown  below. 


web  cam 
4  spot  lights 
mirror 

Kinect  for  PC 


projector 
instrument/suture  tray 


tracker  emitter 
shoulders  trimmed 

laser  cut  acrylic  frame 


Figure  6:  (Left  Lower)  Detailed  design  of  USUHS2013  version  of  system,  complete  with  all  assembly/mating 
features,  additional  lights,  optimal  arrangement  of  imaging  and  projector  components,  revised  instrument/suture 
holder  rack,  revised  placement  of  magnetic  field  emitter.  (Right)  Fully  assembled  system  ready  for  packing  for 
transport  to  Ocular  Trauma  Course. 

This  iteration  of  the  system  implemented  the  following: 

•  Arrangement  of  optical  component  arrangements  to  optimize  overlap  between  fields  of  view  of 
projector,  Kinect,  web  cam  and  lighting  system 

•  Added  additional  light  source  to  increase  operating  field  brightness 
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•  Added  data  acquisition  interface  control  of  lighting  system  brightness  using  the  ''Mind  Board" 
developed  in  Year  1  for  sampling  surgical  instrument  force/grasp  position  signals 

•  Detailed  design  of  panel  components  including  interlocking  t-slot  joining  features 

•  Finalization  of  mating  features  between  base  and  upper  stand  for  separation  and  shipping  with 
minimal  disassembly  of  subcomponents 

•  Trimmed  the  shoulders  of  the  head  form  core  and  silicone  skin  covering  to  permit  fit  within 
structure,  in  turn  designed  to  fit  within  standard  Pelican  cases 

•  Modified  instrument  holder  seats 

•  Laser  cut  acrylic  panels,  learned  to  perform  mirror/glass  cutting,  cut  mirror  to  size 

•  3D  printed  lighting  fixtures,  instrument  holder  seats 

•  Full  assembly  &  testing  of  system  for  optical  alignment 


Figure  7:  Fields  of  view  of  projection/imaging  components.  Highlighted  in  blue  are:  (left)  video  projection,  (center) 
Kinect  motion  tracking,  (right)  web-cam.  Use  of  a  first-surface  mirror  mounted  near  the  top  of  the  system  allows 
for  reduced  overall  height;  minimum  focal  distance  for  Kinect  and  projector  would  otherwise  add  12"  or  more 
vertically. 

Observations  made  and  comments  generated  at  the  2013  USUHS  Ocular  Trauma  course  and  in  a 
meeting  with  Dr.  Frietag  in  late  Q8  were  fewer  than  in  Year  1,  a  promising  change,  signaling  that  the 
system  is  close  to  being  fully  acceptable  by  trainers  for  these  procedures.  Requests  for  future 
modifications,  likely  to  be  made  outside  the  scope  of  this  program  include: 

•  Increase  size  of  storage  tray  over  mannequin  chest 

•  Move  suture  boxes  out  of  the  "sterile  field"  of  the  simulator 

•  Lower  the  tool  rack,  which  will  be  possible  with  the  removal  of  the  suture  box  storage  space 

•  Possible  inclusion  of  a  "swing-out"  tray  for  suture  storage 

•  Modification  of  the  use  model,  such  that  trainees  would  withdraw  the  desired  set  of  sutures  in 
advance,  not  interacting  with  the  storage  boxes  during  a  scenario 

Lastly  for  the  standalone  system,  in  reviewing  the  magnetic  tracker  documentation,  we  found  a  warning 
for  pacemaker  wearers  and  users  of  similar  technology  against  use  of  the  sensor  technology  without 
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consultation  with  their  physician.  To  avoid  potential  risk  to  end  users  of  the  current  system,  we  have 
altered  the  IRB  protocol  to  exclude  pacemaker  users,  and  added  warning  labels  supplied  by  Ascension  to 
the  exterior  of  the  system. 

2.3.4. b.  Force  sensing  strain  gages  for  scissors  and  needle  holders, 
characterization  of  sensing 

In  the  Q6  report,  we  described  the  need  to  add  additional  strain  gages  to  the  scissors  and  needle  holder 
instruments,  for  use  in  better  detecting  contact  and  interaction  with  tissue  and  with  needles.  The 
previously  installed  gages  only  detected  open/closing  action,  without  the  capacity  to  determine  when 
and  what  type  of  object  was  in  contact  with  the  instrument.  Cutting  tissue  presents  a  different  force 
profile  from  cutting  through  suture.  We  presented  finite  element  analysis  plots  showing  optimal 
locations  for  gaging.  Examples  of  curved  scissors  and  needle  holders  are  shown  below. 


Figure  8:  FEA  of  scissors  and  needle  holders  showing  optimal  placement  for  strain  gages;  instruments  showing 
design  positions  of  gages  for  force  sensing  and  inactive  gage  for  temperature  compensation.  Top  to  bottom: 
curved  Westcott  scissors  and  Castroviejo  needle  holder. 
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Figure  9:  Detail  of  scissors  and  needle  holder  with  silicon  strain  gages  installed. 

Described  in  the  Q7  report,  additional  metal  foil  and  silicone  strain  gages  were  added  to  the  scissors  and 
needle  holder  instruments  to  allow  detection  of  relative  stiffness  of  grasped/cut  objects,  for  example  to 
discriminate  between  grasping  of  tissue  vs.  grasping  of  a  suture  needle  vs.  closure  on  empty  space.  This 
distinction  can  be  recognized  by  the  event-based  gesture  recognition  system,  as  the  force-displacement 
response  of  the  instrument  changes. 

In  empty  space,  the  position  sensing  gage  registers  the  user  squeezing  the  instrument  closed,  without  an 
intial  response  from  the  force  sensing  gage,  located  close  to  the  instrument  tip.  Once  the  tips  come 
together,  the  force  sensing  gage  begins  to  respond,  and  the  two  responses  plotted  against  each  other 
generate  a  curve  that  is  related  to  the  instrument's  own  spring  stiffness.  If  an  object  is  grasped,  the 
position  at  which  the  force  response  begins  occurs  earlier  than  for  the  empty  instrument,  giving 
information  about  approximate  size  of  the  object  grasped.  This  shifts  the  break-point  in  the  force- 
displacement  curve  accordingly.  The  slope  of  the  characteristic  curve  also  changes,  as  the  response  is 
now  a  combination  of  the  stiffness  of  the  instrument  and  the  object  grasped. 


stiffen  « - Softer 


ForcepsTheoretkal  Graph 


Figure  10:  Schematic  of  force-displacement  curves  for  use  of  grasping  instruments  in  open  space  and  grasping 
different  stiffness  tissues. 

A  simple  jig  was  designed  to  repeatably  close  the  forceps,  shown  below.  In  testing,  strain  gage  outputs 
were  collected  for  each  full  turn  of  the  lead  screw.  For  scissors  and  the  needle  holder,  digital  calipers 
were  locked  at  a  series  of  closure  positions. 
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Figure  11:  Forceps  force-displacement  test  rig. 

In  the  graphs  below,  taken  using  the  forceps,  the  shifts  in  break  point  and  overall  slope  are  clearly 
shown. 


Figure  12:Normalized  strain  gage  output  vs.  instrument  closure:  red  curve  -  closure  sensor,  blue  curve  -  grasp 
force  sensor.  (Left)  Closing  instrument,  with  tips  coming  together  at  5mm  displacement  of  lead  screw,  with  force 
increasing  as  forceps  are  squeezed  further.  (Center)  Grasping  soft  object,  with  initial  deviation  from  baseline  curve 
at  4mm  displacement,  difference  in  force  slope  with  additional  squeezing.  (Right)  Large,  hard  object,  with  contact 
detected  at  2mm. 

These  same  data  sets  were  collected  for  each  of  the  instruments  planned  for  use  at  the  USUHS  test. 
Plots  showing  force  strain  gage  output  (normalized  with  respect  to  maximum  output  over  all 
instruments)  vs.  normalized  grasping  displacement  (using  lead  screw  jig  or  digital  calipers)  are  shown 
below. 

Different  instrument  types  also  show  different  characteristic  responses.  The  needle  holders  are  of  a 
locking  variety,  with  a  spring-loaded  mechanism  that  holds  the  jaws  closed  after  the  user  has  squeezed 
the  instrument  far  enough.  This  results  in  a  momentary  reversal  of  the  force  curve  at  the  locking  point. 
This  locking  action  also  appears  in  the  pure  closure  position  sensing  response  described  in  earlier 
reports.  Similarly,  the  scissors  show  almost  no  output  force  in  empty  space  until  complete  closure  is 
achieved,  while  force  increases  during  cutting,  as  the  shearing  forces  generated  by  the  scissors  increase 
while  cutting  through  an  object.  In  the  cutting  case,  as  the  scissors  close  the  cutting  point  shifts  away 
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from  the  scissor  axis,  increasing  the  bending  moment  on  the  scissors  (force  x  distance  increases),  and 
the  angle  between  the  cutting  edges  becomes  smaller,  so  a  larger  section  of  the  material  is  being 
sheared.  This  results  in  the  non-linear  increase  in  the  response  observed. 

Described  below,  these  curves  are  being  built  into  the  gesture  recognition  architecture,  however  have 
not  been  thoroughly  tested  as  of  our  design  freeze  point  for  the  USUHS  testing,  so  the  earlier,  position- 
based  event  detection  algorithms  will  be  used  to  ensure  reliability. 


Force  Comparison  (base  on  opening)  Force  Comparison  (base  on  opening) 


Figure  13:  Strain  gage  response  curves:  (upper  left)  0.5mm  jaw  grasping  forceps;  (upper  right)  0.3mm  jaw  grasping 
forceps;  (center  left)  large  jaw  needle  holder;  (center  right)  small  jaw  needle  holder;  (lower  left)  curved  scissors. 

Revision  of  instrument  connector  shells,  stabilization  of  instrument/sensor  orientation 

In  Q7  we  began  work  to  replace  the  earlier  iteration  of  the  housing  that  connects  the  Ascension  sensor, 
the  strain  gage  amplifiers  and  the  instruments  together.  This  version,  used  at  the  2012  USUHS  eye 
trauma  course  data  collection  exercise,  was  subject  to  accidental  misalignment  of  the  connector,  which 
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prevented  proper  mating  and  distraction  to  the  surgeon/trainee.  The  new  design  includes  spiral  ''ramp'' 
features  to  guarantee  that  during  plug  insertion,  the  instrument  and  plug  rotate  with  respect  to  each 
other  in  the  correct  direction  towards  the  correct  connector  alignment.  In  addition,  spherical  features 
allow  for  mating  with  the  instrument  holding  rack  in  any  orientation  rotated  around  the  instrument's 
axis. 
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Figure  14:  New  design  for  instrument/sensor  interconnect  format  with  features  to  make  insertion/removal 
intuitive  and  resistant  to  misalighment.  For  scissor/needle  holder  instruments,  FEA  of  concepts  for  flexural  beams 
for  holding  connector/sensor  housing  aligned  with  long  axis  of  each  instrument.  Not  necessary  for  forceps. 

These  housings  were  customized  for  all  of  the  available  instruments.  For  the  scissors  and  needle 
holders,  the  pivoting  action  of  the  proximal  ends  allowed  the  housing  to  rotate  away  from  the  axis  of 
the  instrument,  causing  position  sensing  to  miscalculate  the  tip  location.  To  prevent  this,  stabilizing 
cantilever  flexures  were  designed  into  the  housings  to  hold  the  scissor/needle  holder  arms  apart.  Early 
versions  were  printed  as  part  of  the  housing  using  the  Polyjet  process  of  our  Objet  rapid  prototyping 
system.  This  material  suffers  from  creep  deformation  when  held  under  continuous  load  (e.g.  needle 
holder  locked  closed);  a  solution  was  found  in  the  form  of  combining  parts  from  both  of  our  RP  systems 
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-  the  high  resolution  Polyjet  parts  for  the  rigid  elements  of  the  housing,  with  a  flexible  cantilever 
fabricated  using  our  FDM  system,  which  uses  a  different  plastic  that  appears  to  be  substantially  less 
subject  to  creep. 

Over  the  course  of  Q7,  variations  of  the  flexure  were  designed  for  use  on  both  needle  holders  and  the 
two  large  scissors  used  for  the  lid  laceration  scenario.  Versions  of  the  housing/flexure  combinations  for 
the  corneo-scleral  scissors  are  pending  and  will  be  complete  by  the  time  the  globe  surgery  modules  are 
complete. 


Figure  15:  Design  and  completed  versions  of  housings  with  stabilizing  flexures  on  needle  holder  and  scissor 
instruments. 

In  Q8,  in  advance  of  the  USUHS  testing,  we  tested  the  instruments  through  normal  use  and  calibration 
to  determine  if  there  were  any  failure  modes  that  could  be  identified  and  resolved  before  the  testing. 
We  found  that  the  scissor  flexure  design  is  robust,  as  is  that  of  the  small-jaw  needle  holder.  One  of  the 
flexures  of  the  large  jaw  needle  holder  (upper  instrument  in  figure  above)  split  off  at  the  base,  so  the 
cantilever  design  was  revised  to  reduce  strain  generated  at  that  point.  Spares  of  all  of  the  flexures  were 
prepared  in  advance  of  user  testing,  in  case  of  unexpected  failures. 

The  attachment  clips  that  mate  the  needle  holders  and  scissors  to  the  miniPCB  housing/connector  shells 
were  modified  for  all  of  the  instruments  to  improve  alignment  of  the  tool  tip  to  the  axis  of  the  housing  - 
when  the  jaws  are  closed,  slight  differences  in  the  arm  lengths  of  the  instruments  can  cause  the  jaws  to 
deviate  to  one  side  or  the  other  of  the  housing  axis.  Adjusting  the  design  of  the  attachment  clips, 
attaching  to  slightly  more  or  less  of  the  end  of  the  arms,  allows  correction  of  this  deviation. 

During  testing  of  the  large  jaw  needle  holder,  we  found  that  grasping  a  needle  at  the  throat  of  the  jaws 
generates  substantial  strain  in  the  arm  where  the  silicon  strain  gages  that  we  had  ordered.  This  resulted 
in  the  failure  of  one  of  those  gages.  We  identified  normal  strain  gages  small  enough  for  mounting  on 
the  inside  surfaces  of  the  needle  holder  arms,  near  the  pivot  point  and  installed  a  replacement.  We  also 
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added  these  same  small  strain  gages  to  the  equivalent  surfaces  of  the  scissors  to  detect  cut  forces. 
Calibration  is  described  above,  and  interfacing  with  the  software  is  described  below. 

Following  the  USUHS  trials,  based  on  real-world  use  observations,  we  began  to  change  gain  values  in  the 
force  sensor  amplifier  boards  to  match  the  range  of  sensitivity  of  the  instruments  to  the  force  range 
actually  used  by  physicians,  which  was  much  smaller  than  what  we  employed  during  our  testing. 


2.3.4.C.  Polhemus  sensor  evaluation  completion 

The  Q7  report  described  the  fabrication  and  initial  testing  of  alternate  position/orientation  tracking 
systems  from  Polhemus,  which  resulted  from  contact  established  with  the  company  at  the  TATRC  booth 
at  MMVR  earlier  this  year.  During  Q8,  we  completed  testing  of  the  sensors  and  returned  the  loaned 
system  back  to  the  vendor.  We  found  the  increased  cable  flexibility  to  be  a  desirable  feature,  vs.  the 
Ascension  Technologies  system  that  we  currently  use,  however  the  sensor  size  is  too  large  for  the 
ophthalmic  instruments  and  we  observed  additional  noise  added  to  the  strain  gage  sensor  signals.  We 
held  discussions  with  Ascension  and  evaluated  material  samples  in  regards  to  the  possibility  of  creating 
cabling  with  enhanced  flexibility  and  with  integrated  conductors  to  support  our  force/grasp  strain  gages, 
however  at  this  time  we  have  not  pursued  desirable  modifications  as  the  expense  of  the  engineering 
work  is  not  justified  for  a  relatively  small  improvement  to  our  prototype.  We  will  recommend 
reexamination  of  this  feature  in  any  commercialization  discussions. 


Figure  16:  (left)  designs  for  Polhemus  sensor  holders  for  teardrop  sensor  style,  with  standard  and  thin  cable 
implementations,  (right)  Sensor  cabling  implemented  for  physical  testing. 

The  Augmented  Reality  microscope  design,  described  in  detail  in  later  sections,  was  modified  to  support 
use  of  Polhemus  sensors,  such  that  designs  would  be  available  should  commercialization  discussions 
result  in  a  shift  from  the  Ascension  to  the  Polhemus  sensor  system. 
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Figure  17:Designs  for  original  Ascension  sensor  for  microscope  position  tracking,  design  revision  for  Polhemus 
sensor  and  modified  shell  design 


2.3.5.  “Month  7”  - 

2. 3. 5. a.  Design  of  skeletal  structure,  docking  points,  sensor/actuator  interfaces 

with  ocular  and  craniofacial  trauma  modules. 

Eye  motion  mechanism 

During  Q5,  building  on  earlier  design  concepts,  masters  student  intern,  Simone  Zivieri  designed  and 
prototyped  a  mechanism  that  includes  all  of  the  globe  motion  features  that  will  be  included  in  the  eye 
modules:  eye  pitch  and  yaw,  proptosis,  backdriveability  in  pitch  and  yaw  (to  allow  the  surgeon  to 
manipulate  the  eye)  and  applied  force  sensing  to  detect  pressure/load  applied  to  the  globe  and  the 
reduction  of  force  on  the  eye  expected  on  successful  canthotomy/cantholysis. 

The  mechanism  shown  below  is  the  first  iteration  which  supports  the  mechanical  m  otions  and 
compliance  features  analogous  to  normal  eye  motions,  and  relative  sizes  of  the  components  in 
reference  to  our  CT-derived  skull  model. 


28 


Figure  18:  Eye  motion  module  for  pitch,  yaw  and  proptosis,  with  backdriveability  in  globe  motion.  Sizing  of 
components  relative  to  skull  model. 

Eyeball  baseplate  redesign 

To  allow  for  attachment  of  our  replaceable  eye  globe  models  and  provide  fluid  and  signal  connectivity, 
we  have  included  fluid  channels  that  protrude  into  the  space  of  the  globe.  These  serve  the  dual  purpose 
of  providing  mechanical  connection  through  a  tight  friction  fit.  The  fluid  channels  will  connect  the  globe 
to  a  fake  blood  supply  for  simulation  of  bleeding  and/or  hemorrhagic  chemosis  -  bleeding  behind  the 
conjunctiva  creating  a  red  zone  around  the  cornea,  one  of  the  common  signs  of  retrobulbar 
hemorrhage.  They  will  further  provide  the  fluid  source  for  testing  for  suture  watertightness  and  likely 
control  of  the  pupil  mechanism.  Shown  below  are  modifications,  including  reduction  in  thickness  (to 
accommodate  large  angular  motion  without  collision  between  fluid  connections  and  the  supporting 
post),  the  fluid  channels  with  connection  points  front  and  back  and  corrections  to  the  central  spherical 
joint  design  from  the  earlier  version.  It  also  integrates  a  first  test  at  a  revision  to  the  elastic  member 
connection,  replacing  the  latex  rubber  bands  with  neoprene  o-ring  material  for  improved  reliability.  This 
is  expected  to  increase  the  lifetime  of  the  mechanism  (the  neoprene  is  expected  to  have  a  longer 
lifespan  than  latex,  which  rapidly  dries  and  fractures).  The  attachment  mechanism  is  no  longer  subject 
to  the  possibility  of  the  elastic  member  becoming  dislodged,  as  set  screws  maintain  the  connections. 
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Figure  19:  base  plate  modifications,  rapid  prototyped  test  versions  of  plate  and  globe  showing  plate  and  globe 
segment  separately  and  assembled.  Features  include  bosses  for  neoprene  cords  to  replace  latex  rubber  bands. 
Modified  elastic  transmission  components:  o-ring  cord  threaded  through  servo  arm,  initial  baseplate  attachment. 

During  Q6,  examination  of  sections  through  the  model  of  the  skull  was  done  to  verify  mechanism  fit 
within  the  skull  model,  and  maintain  a  reasonable  thickness  of  skull  structure  external  to  the  motion 
mechanism.  The  initial  orthogonal  arrangements  of  eye  motion  mechanisms  would  violate  the  outer 
section  of  the  skull,  so  they  were  angled  inwards  and  the  servo/cord  arrangements  and  contour  of  the 
globe  base  plates  modified  to  retain  the  normal  forward  looking  orientation  of  the  globe. 


Figure  20:  Sectioning  of  skull  with  prototype  motion  mechanism  showing  excursion  of  elastic  cords  from  surface  of 
skull  in  zygomatic/sphenoid  region. 

Proptosis  mechanism 

The  proptosis  mechanism  designed  early  in  Q5  reached  a  functional  test  model  by  the  end  of  the 
quarter,  demonstrating  a  range  of  motion  of  approximately  10mm,  more  than  sufficient  to  generate  the 
protrusion  of  the  globe  necessary  to  indicate  retrobulbar  hemorrhage.  Shown  below  is  proof  of 
principle  design,  and  the  sliding  carriage  shown  in  the  extended  configuration. 
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Figure  21:  Proptosis  mechanism  (separate  from  pitch/yaw  components) 

The  upper  part  of  the  mechanism,  the  components  that  drive  proptosis,  remains  in  a  straight  direction, 
in  order  to  keep  the  translation  of  the  eye  the  same  as  before.  Changing  from  FMA  PS20  servos  to 
smaller  Hitech  HS-5035HD  models  allowed  unification  of  multiple  components  of  the  servo  support 
frame  into  one  part.  Flipping  the  orientation  of  one  servo  of  each  pair  further  reduces  size  of  the 
mechanism  and  continues  to  prevent  collision  between  the  control  arms.  This  change  enables 
mechanism  rotation  of  15  degrees  from  the  original  position  to  align  with  the  orbit  axis  and  preserve  the 
orbital  wall  thickness.  A  prototype  of  the  compact  version  of  the  eye  motion  system  was  built,  shown 
below.  The  system  has  been  tested  with  positive  results. 
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Figure  22:  comparison  of  initial  and  modified  orientations  of  eye  motion  mechanisms,  preserving  orbital  wall 
thickness  and  reducing  overall  mechanism  size.  Assembled,  compact  motion  module. 

Blinking  mechanism 

In  Q6,  the  final  main  actuated  component  of  the  eye  motion  system,  the  blinking  mechanism  was 
prototyped,  so  that  the  mannequin-mounted  version  of  the  system  will  provide  a  sense  of  presence  and 
indication  of  level  of  consciousness,  and  help  to  provide  indications  of  eye  trauma;  blinking  would  be 
prevented,  for  example,  during  a  retrobulbar  hemorrhage  scenario,  for  example. 

The  eyelid  is  made  of  silicone  soft  layers  (see  also  below  for  portal  design),  mounted  to  an  orbital  rim 
structure.  A  silicone  protrusion  of  the  eyelid,  corresponding  roughly  with  the  levator  and  Muller's 
muscles,  will  pass  through  a  gap  created  in  the  upper  part  of  the  orbital  rim  of  the  portal.  This 
protrusion  will  link  with  a  retraction  mechanism,  part  of  the  eye  motion  assembly.  Shown  below  is  Q6 
iteration  of  the  design  of  the  bony  components  of  the  system,  with  protrusion  slots. 
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Figure  23:  Orbital  rim  installed  in  skull,  with  slot  for  "levator  muscle"  protrusion  of  eye  lid;  orbital  rim  removed 
with  slot  for  access  to  blinking  mechanism  behind  forehead. 

The  blinking  mechanism  is  a  servo-actuated  crank-slider,  with  an  end-hook  which  will  mate  with  the  eye 
lid  protrusion.  A  crank-slider  linkage  is  necessary  to  orient  the  pulling  direction  of  the  eyelid  and  to 
avoid  collision  with  the  internal  wall  in  the  skull.  The  final  hook  shape  will  be  determined  as  we  develop 
the  removable  portal  in  more  detail. 


Figure  24:  Blink  motion  eyelid  retraction  mechanism.  Skull  segment  with  motion  mechanism  for  testing  of 
assembly  and  fit.  Also  shown  is  partially  seated  bony  orbital  rim  which  will  support  the  trauma  portal  soft  tissue. 


Replication  for  left  and  right  sides 

Also  in  Q6,  the  bilateral  system  was  designed.  A  simple  reflection  of  the  mechanism  across  the  sagittal 
plane  results  in  collision  between  the  servo  control  arms  of  the  posterior  servo.  Instead,  keeping  the 
same  servo  mounting  frame  and  rotating  its  placement  prevents  collisions  and  reduces  part  count.  This 
can  be  seen  in  the  detail  of  Figure  25. 


The  slide  mechanisms  for  proptosis  have  been  lengthened,  their  anterior  ends  modified  to  support  the 
angled  orientation  of  the  mechanism,  and  the  main  frame  that  supports  the  assembly  widened  with 
mounting  features  for  attachment  to  the  skull.  To  further  unify  the  design,  the  same,  smaller  servo  used 
for  eye  motion  is  now  used  for  the  proptosis  motion.  This  design  decision  simplifies  the  bill  of  materials, 
also  reduces  mechanism  size  and  simplifies  software  control  of  the  system. 


Two  additional  servos  are  shown,  which  will  be  used  for  the  blinking  mechanism,  described  below. 
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Figure  25:  Bilateral  eye  motion  mechanism  with  servo  configuration  maintained  left/right  vs.  reflected.. 


Figure  26:  Partitioning  of  skull;  skull  component  with  motion  cavity  shown;  assembled  skull  design  with 

mechanism. 

During  Q7,  some  modifications  were  made  following  the  work  of  our  intern,  Mr.  Zivieri,  mainly  in  terms 
of  altering  clearances  to  avoid  binding  of  the  proptosis  motion,  enclosing  moving  parts  to  protect  them 
from  damage  and  allow  for  later  addition  of  lubricating  grease,  and  simplifying  fabrication  by  combining 
previously  separate  parts  held  together  by  screws,  into  monolithic  components  that  take  better 
advantage  of  the  RP  facilities  available. 

The  new  assembly  was  connected  to  our  drive  circuitry  for  testing  and  debugging  of  the  code,  mainly  to 
ensure  that  all  of  the  independent  and  linked  motions  behave  correctly  -  blinking  of  both  eyes  occurs  in 
the  same  direction  (the  servos  are  arranged  opposite  to  each  other,  so  inverse  command  signals  must 
be  sent  to  them  for  the  output  shafts  to  generate  the  same  blinking  motions). 
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Eye  motion  kinematics 

The  eye  mechanism  can  rotate  in  both  the  yaw  (a)  and  pitch  ((3)  directions,  under  the  control  of  the  two 
independent  servos,  with  rotations  0i  and  02.  Because  the  elastic  cables  would  physically  interfere  with 
rigid  components  of  the  system  if  the  servos  were  oriented  at  90  degrees  to  each  other,  the  servos  are 
rotated  50  degrees  away  from  parallel,  or  +/-  25  degrees  away  from  the  vertical  y-axis  shown  in  Figure 
27.  In  this  configuration,  rotation  of  the  globe  requires  motions  of  both  servos. 


Figure  27:  Eye  motion  polar  coordinate  system.  Globe  motion  control  is  defined  about  the  X  and  Y  axes,  while  the 
servos  rotate  at  offset  axes,  with  motions  0i  and  02. 

To  make  the  calculations  more  general,  the  servo  offset  angle  is  defined  as  y  (instead  of  using  25 
degrees  specifically)  and  the  command  positions  sent  to  the  servos  for  a  given  eye  are  determined  by:: 


01  =  tan  ^ 


(sin  a  cos  y  +  cos  a  sin  p  sin  y) 
cos  a  cos  P 
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^2 


tan  ^ 


and 

(sin  a  cos  y  —  cos  a  sin  p  sin  y) 
cos  a  cos  P 


These  equations  have  been  implemented  in  the  motion  controller  for  testing  the  eye  motion 
mechanism.  A  basic  point-and-click  interface  was  generated  to  control  both  eye  orientation  and 
proptosis  position,  as  well  as  motion  speed. 


Motion  controller  test  user  interface 

Using  an  Arduino  2009  microcontroller  and  a  Processing  graphical  user  interface,  motion  control  of  the 
mechanism  was  achieved.  This  formed  the  basis  of  a  later,  more  comprehensive  controller,  described 
below.  Examples  of  control  entry  and  corresponding  eye  mechanism  positions  are  shown. 


n 


a=-30,  p=0 


a=0,  p=+30 


proptosis:  +14mm 


During  Q7,  we  described  a  move  from  the  Arduino  2009  platform  for  interfacing  with  the  eye/head/neck 
control  systems  to  the  Arduino  Mega  2560.  The  new  board  supports  40  more  digital  I/O  lines  and  10 
additional  analog  input  channels.  Based  on  the  restrictions  of  the  central  microprocessor  and  our  needs 
for  controlling  multiple  functions,  we  replaced  a  stock  interface  program  available  for  the  Arduino  with 
one  of  our  own,  which  optimizes  the  available  I/O  channels  for  our  use.  The  board  is  now  set  up  to  run 
8  different  latching  valves,  12  servos  (primarily  eye  mechanism  motion),  I2C  communications  to  the 
digital-to-analog  modules  for  use  with  the  electronic  regulators  we  acquired  earlier,  one  or  two  stepper 
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motors,  two  quadrature  encoders,  and  unassigned  I/O  for  10  PWM  or  digital  I/O  lines  (e.g.  pump 
control,  single  acting  valves),  three  serial  communication  ports,  and  all  16  analog  input  lines. 

To  test  this  system  and  provide  control  for  the  existing  hardware  elements,  we  prepared  the  GUI 
currently  being  used  with  the  eye  motion  platform  described  above  and  the  latching  valves  on  the  air 
muscle  test  bed  developed  in  earlier  quarters  (see  neck  motion  sections,  below). 


Figure  28:Comprehensive  development  graphical  user  interface,  including  pneumatic  controls  (upper  left), 
proptosis  position  and  speed  for  left  and  right  eye  (upper  right),  blinking  servo  position  and  speed  and  whether 
left/right  eyes  are  linked  (middle  right),  and  independent  or  linked  eye  orientation  control 

The  control  circuitry  was  unified  as  well,  shifting  from  an  initial  version  with  discrete  transistors  to  a 
smaller  number  of  ICs.  A  Bluetooth  module  was  added  to  the  circuit,  which  allows  the  Instructor  Tablet 
computer  associated  with  the  mannequin  to  control  the  new  components  wirelessly.  When  connected 
to  the  mannequin's  power  supply,  we  have  achieved  a  state  where  the  new  components  require  no 
external  wired  connections. 

Testing  of  the  valves  in  late  Q7  showed  that  the  12V  mannequin  supply,  when  run  through  the  earlier 
discrete  transistor  circuit  or  the  new  version,  did  not  produce  sufficiently  high  voltage  at  the  valve 
inputs,  which  are  also  rated  at  12V.  Voltage  drops  across  the  transistors  result  in  insufficient  voltage  at 
the  valves,  so  we  have  observed  that  some  of  the  valves  are  not  reliably  actuated.  To  solve  this  issue, 
we  ordered,  installed  and  tested  a  DC/DC  converter,  which  boost  the  12V  supply  from  the  mannequin  to 
15V  output  of  the  converter.  This  comes  at  the  cost  of  a  higher  current  draw  from  the  mannequin,  so 
we  are  limited  to  running  two  valves  at  a  time  at  most  (mannequin  current  limit  is  lA).  Since  the  valves 
would  generally  be  activated  one  or  two  at  a  time  during  normal  operation,  this  is  not  an  issue  -  only 
during  initialization  of  the  system  would  numerous  valves  be  activated  in  short  order,  but  since  the  time 
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to  pulse  the  valves  to  set  their  state  is  under  lOOms,  this  does  not  present  problems.  Modifications  to 
the  custom  interface  code  to  support  easy  sequencing  of  multiple  valves  are  in  process. 


Eye  motion 
servo  control 

Digital-to-analog 
converter  for 
regulator 

Bluetooth 

module 

Electronic 

regulator 


Valve  control 
circuit 


Arduino  Mega 


Figure  29:  Protoboard  for  eye  servo  and  neck  pneumatic  valve  control 


Eye  globe  design  iteration 

In  Year  1,  our  earlier  globe  models  provided  the  basis  for  the  development  of  molding  techniques  and 
materials  selection,  including  the  use  of  the  nylon  mesh  to  enhance  suture  retention  and  rip-stop 
characteristics  in  the  silicone.  During  Q5,  we  modified  the  globe  design  for  compatibility  with  the 
mobile  eye  platform  (the  earlier  versions  were  fixed  in  location)  and  to  add  the  bulbar  conjunctiva, 
which  was  previously  not  a  feature.  Taking  advantage  of  the  capabilities  of  the  rapid  prototyping 
system,  we  created  a  one-piece  design  to  test  fabrication,  attachment  and  retention  concepts  for  the 
new  globe.  One  of  the  two  fluid  channels  enters  the  globe  and  then  exits  into  the  space  between  the 
globe  and  the  conjunctiva,  providing  a  means  for  filling  the  space  with  fake  blood  to  simulate 
hemorrhagic  chemosis.  Future  steps  were  envisioned  as  developing  a  fabrication  sequence  for  silicone 
molding  of  this  structure,  addition  of  muscle  insertions  into  the  globe  and  refinement  of  the  geometry 
of  the  conjunctiva  component. 


Figure  30:  Front  and  back  views  of  new  globe  concept  showing  conjunctival  layer  and  fluid/retention  ports. 
Concept  of  attachment  to  baseplate  and  fluid  channels. 
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During  Q6,  this  design  evolved  into  a  two  part  assembly:  one  for  the  eye  lid  and  orbital  rim,  one  for  the 
globe.  This  approach  will  permit  mixing  and  matching  of  different  kinds  of  trauma  modules,  from  simple 
lid  laceration  with  no  globe  involvement,  to  lid  avulsion  and  lacerated  orbit.  The  globe  component  must 
be  compatible  with  the  symptoms  of  retrobulbar  hemorrhage,  one  of  which  is  hemorrhagic  chemosis  - 
bleeding  underneath  the  bulbar  conjunctiva  layer.  Shown  below  is  the  bony  orbital  rim  described  above 
overlying  a  multi-layer  flexible  conjunctiva  structure.  The  blue  layer  is  part  of  the  globe  module,  bonded 
to  the  globe  at  the  margin  of  the  cornea,  extending  to  the  seat  in  which  the  orbital  rim  is  mounted.  This 
would  be  semi-transparent,  so  simulated  blood  would  be  visible.  The  red  layer  is  mounted  to  the  skull 
and  seals  the  fluid  space  between  red  and  blue  apart  from  the  mechanical  and  electrical  spaces  within 
the  skull.  The  tubes  and  ports  that  connect  to  the  globe  (described  in  previous  reports)  provide  an  inlet 
for  simulated  blood  to  enter  this  space.  Removal  of  the  eye  lid  and  globe  portals  will  provide  access  to 
the  fluid  space  for  ease  of  cleaning  and  inspection. 


Figure  31:  Concepts  for  flexible  conjunctiva  layers  and  sealing  surface  to  protect  mechanisms  from  fluid  contact. 

During  Q8  we  experimented  with  a  technique  for  producing  a  conjunctival  layer.  The  layer  will  consist  of 
a  hollow  sphere  of  0030  durometer  silicone  with  a  wall  thickness  no  greater  than  .02".  The  current 
methods  for  creating  this  part  are  1)  a  dipping  process  around  a  hollow  meltable  core  and  2)  a  glove 
mold.  The  benefits  of  the  dipping  process  are  speed  and  simplicity— the  drawback  being  poor  control 
over  thickness.  The  advantage  of  the  glove  mold  is  that  wall  thickness  and  overall  shape  can  be 
controlled  to  a  very  precise  degree,  but  the  castings  must  be  produced  at  high  pressure  to  ensure  that 
the  mold  is  filled  without  any  air  pockets.  The  extreme  thinness  of  the  part  will  result  in  many  casting 
failures.  The  conjunctiva  will  be  attached  to  the  edge  of  the  cornea  with  silicone  glue  and  will  also  be 
attached  to  the  inside  of  the  eyelid  with  silicone  glue.  The  opening  of  the  eyelid  will  serve  as  a  guide  for 
the  cut  line  of  the  conjunctiva. 

Eyelid  anatomy  development 

During  Q5,  we  began  to  add  silastic  tubing  into  the  margin  of  the  eyelid  model  to  provide  the  anatomy 
for  performing  canalicular  repair,  for  more  advanced  eye  lid  laceration  scenarios  that  the  initial  full¬ 
thickness  marginal  laceration.  Dr.  Mazzoli  provided  advice  and  detail  on  the  relative  sizes  of  the 
canaliculi  and  the  punctum,  the  small  opening  that  drains  tears  from  the  eye.  We  have  acquired  larger 
tubing  than  the  initial  0.5mm  ID  version  and  are  developing  methods  for  properly  retaining  it  in  the 
molds. 
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In  addition,  we  began  to  CT-derived  skin  surface  models  to  generate  new  eye  lid  structures,  vs.  our 
earlier  Phase  I  system  that  used  the  life-cast  original.  Shown  below  are  surfaced  CT  data,  CAD  models  of 
elements  of  the  mold,  partially  molded  lid  structures,  with  the  first  mold  lid  removed  prior  to  installation 
of  the  second  lid  with  the  cavity  for  the  tarsal  plate,  and  the  front  and  back  views  of  the  new  lid 
structures. 


Figure  32:  Eyelid  based  on  CT-derived  data:  (top  row)  Mimics  software  view  of  anatomy,  CAD  model  of  mold, 
partially  completed  eye  lid,  prior  to  over-molding  of  tarsal  plate,  (bottom  row)  Front  and  back  views  of  overmolded 
portal  structure,  CAD  model  with  laceration,  physical  test  model  with  defined  laceration  molded-in. 

During  Q6,  we  reported  on  the  direct  design  of  a  laceration  into  the  3D  printed  molds,  so  that  variation 
in  the  trauma  modules  due  to  artistic  variation  would  not  occur.  Using  the  components  that  were 
produced  at  that  time,  we  performed  tests  to  determine  how  to  create  molded  eye  lids  with  multiple 
layers  that  would  be  compatible  with  the  blinking  mechanism  described  above. 

The  initial  part  of  the  test  was  to  create  incisions  in  the  lids  and  attach  threads  to  examine  the  likely 
motion  during  retraction.  These  tests  suggest  that  an  approach  to  the  soft  tissue  components  is  to 
incise  the  skin  layer,  seat  the  molded  soft  tissue  into  a  second  mold  with  the  lid  in  a  hyper  extended 
position  and  mold  in  additional  silicone  to  create  the  redundant  tissue  that  Dr.  Mazzoli  described  for  us 
in  earlier  conversations.  This  redundant  tissue  rolls  over  during  the  eye  lid  opening  process,  creating  the 
fold  of  the  eye  lid.  Using  the  3D  models  extracted  from  the  CT  scans  using  Mimics  and  3-matic  an  inner 
layer  is  designed  that  extends  through  a  cutout  above  the  upper  eyelid  and  an  extruded  feature 
extending  upward  from  the  tarsal  plate.  Design  of  the  hook-mate  to  the  mechanism  described  above 
remains  pending.  Initial  testing  of  a  modified  module  demonstrating  blinking  under  servo  control  is 
described  in  section  2.3.9. 
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Figure  33:  Blinking  soft  tissue  test  object;  design  for  eye  lid  with  levator  nnuscle  structure;  test  version  installed  in 
skull  section,  with  orbital  rinn  and  assennbled  soft  tissue  layers. 

During  Q7,  work  on  moving  the  lacrimal  duct  structures  forward,  necessary  to  complete  development  of 
an  eye  module  for  lacerated  lid  with  canalicular  repair,  the  second  major  eye  trauma  portal  for  this 
project. 

The  lacrimal  assembly  consists  of  the  canalicular  ducts  and  the  nasolacrimal  sac/duct.  With  the  ducts 
running  through  the  margin  of  the  upper  and  lower  eyelids,  the  portal  mold  now  includes  alignment 
slots  for  the  canalicular/tarsal  assembly.  The  ducts  consist  of  silastic  tubing  of  an  appropriate  diameter, 
bonded  and  sealed  with  silicone  glue.  The  assembly  is  then  attached  to  the  tarsal  plate  using  nylon 
string.  The  whole  assembly  is  then  plugged  into  the  revised  mold  using  the  new  alignment  slots.  Sprues, 
which  are  a  by-product  of  the  molding  process  and  are  normally  discarded,  are  preserved  to  serve  as 
alignments  pegs.  The  mold  revision  ensures  proper  placement  of  the  assembly  in  the  mold  prior  to 
injection.  The  entire  assembly  is  then  encapsulated  by  silicone  during  the  casting  of  the  portal,  resulting 
in  a  lifelike,  multilayered  object. 
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Figure  34:(Left)  eye  tarsal  plate,  with  attached  lacrimal,  nasolacrimal  duct  tubing,  reinforcing  mesh  (as  per  previous 
versions  of  the  portal)  (Right)  Mold  for  eyelid  portal,  with  tarsal  plate,  ducts  in  place  for  over-molding  with  softer 
durometer  silicone  for  eyelid/skin  structures.  Sprues  from  tarsal  plate  molding  process  hold  the  plate  and  duct 
structure  in  place  for  overmolding  process. 

During  Q8  we  implemented  and  improved  upon  the  canalicular  structures.  The  overall  size  and  position 
of  the  silastic  tube  was  modified  to  more  accurately  reflect  the  anatomical  structures.  Small  grooves, 
similar  to  the  ones  present  in  an  actual  lateral  canthal  ligament,  have  been  replicated  in  the  module  and 
securely  hold  the  canaliculus  in  place. 

We  have  made  a  substantial  redesign  for  the  tarsal  plate  part  for  use  in  a  second-generation  module, 
which  were  shown  to  our  SME(s)  at  the  2013  USUHS  Ocular  Training  Course.  Unlike  the  previous 
generation  tarsal  plate  part,  a  one-piece,  flat  band  that  resembles  a  simplified  version  of  the  anatomical 
structure,  the  second-generation  tarsal  plate  more  closely  resembles  the  real  anatomy.  It  includes  a 
well-defined  medial  canthal  ligament  which  contains  the  aforementioned  canalicular  grooves  in  the 
superior  and  inferious  crus,  an  anatomically  accurate  lateral  canthal  ligament,  Whitnal's  ligament, 
Lockwood's  ligament,  and  the  upper  and  lower  tarsal  plates. 


Figure  35:(Left)  Textbook  anatomy  of  tarsal  plates,  periorbital  ligaments  used  as  reference  for  advanced  module 
design.  (Right)  Molds  and  cast  version  of  new  anatomy,  fabricated  in  stiffer  silicone  than  surrounding  eyelid  tissue. 

All  of  these  structures  are  cast  as  a  single  part.  The  part  aids  in  providing  a  realistic  surgical  experience, 
strengthens  the  overall  structure  by  providing  more  surface  area  for  the  module  to  bond  with,  and 
provides  a  robust  framework  for  the  attachment  of  the  canalicular  tubes.  Furthermore,  the  lateral 
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canthal  ligament  together  with  the  Lockwood's  ligament  have  been  outfitted  with  wires  to  serve  a  cut 
detectors  during  the  proptosis  procedure.  During  this  procedure,  a  surgeon  is  required  to  cut  through 
the  lateral  canthal  ligaments,  bisecting  it  until  they  reach  the  Lockwood's  ligament.  They  then  bisect  the 
Lockwood's  ligament  until  pressure  on  the  orbit  is  released.  We  can  now  detect  if  these  incisions  have 
been  properly  executed  by  monitoring  a  voltage  passed  through  the  embedded  wire. 

The  electronics  for  this  detection  circuit  are  deliberately  simple;  the  wires  through  the  ligaments  tie  a 
digital  input  channel  on  the  Arduino  microcontroller  (or  other  later  system)  to  ground  (also  providing 
electrical  safety  as  there  is  little  current  passing  through  the  wires  at  any  time).  When  the  wires  are  cut, 
the  ground  connection  is  lost  and  a  pull-up  resistor  internal  to  the  microcontroller  raises  the  measured 
voltage,  signaling  the  incision. 


Figure  36:  (upper  row)  Front  and  rear  views  of  new-version  trauma  module  (uninjured  version).  Soft,  pigmented 
silicone  and  hard,  clear  version  for  illustration.  Internal  structures  include  tarsal  plate  with  ligaments,  periorbital 
fat  deposits,  arterial  vasculature,  canaliculi  and  naso-lacrimal  duct.  Canthal  ligaments  have  fine  wire  embedded  in 
them  (visible  in  rear-view,  bare  wires  and  attached  to  temporary  connector),  arterial  system  has  inlet  and  outlet 
ports  for  blood  simulant  flow  and  external  pressure  sensing,  (lower  row)  Module  connected  to  microprocessor 
showing  illuminated  LEDs  that  indicate  un-cut  anatomy. 

The  module  has  been  further  augmented  by  the  inclusion  of  a  layer  of  periorbital  fat  and  a  lacrimal 
gland.  The  fat  adds  to  the  overall  realism  of  the  module,  creating  a  more  lifelike  feel.  Additionally,  the 
fat  could  be  used  to  simulate  a  subconjuctival  prolapse.  It  is  possible  to  view  the  fat  deposits  as  well  as 
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the  lacrimal  gland  by  inverting  the  eyelid  with  a  forceps.  These  structures  add  a  level  of  realism  that  with 
previously  unavailable. 


Lowsroyelid 
oemral  iai  pocket 


(Left)  Textbook  anatomy  showing  fat  deposits,  lacrimal  gland.  (Right)  Intermediate  step  in  advanced  module 

casting,  showing  placement  of  periorbital  fat  deposits. 

We  have  further  augmented  the  module  by  the  inclusion  of  a  functional  arterial  system.  An  analog  of 
the  authentic  orbital  arterial  system  has  been  designed  and  created  for  the  purpose  of  providing  the 
option  of  a  lifelike  bleeding  scenario.  The  arteries  are  filled  with  a  synthetic  blood  which  is  fed  from  an 
IV  bag.  The  pressure  of  the  arterial  system  will  be  monitored  by  an  external  pressure  sensor.  When  an 
artery  is  cut,  causing  a  hemorrhage,  the  pressure  drop  can  be  detected  by  the  pressure  sensor.  This  data 
can  be  incorporated  into  the  model  at  a  future  period.  We  can  also  detect  when  a  hemorrhage  is 
stopped  by  the  user,  opening  up  the  possibility  for  different  scenarios  than  were  previously  possible. 


Figure  37:(Left)  Textbook  anatomy  showing  arterial  network.  (Center)  Inlay  of  wax  vessels  to  preserve  vessel 
lumens  -  wax  is  melted/dissolved  out  after  silicone  cures  to  produce  empty  vessel  lumens.  (Right)  fully  assembled 
advanced  module,  with  inlet  and  outlet  taps  for  bleeding  from  eyelid  laceration  module. 
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After  several  discussion  with  SMEs,  it  was  determined  that  it  would  be  extremely  difficult  and  not  useful 
to  excise  the  canaliculus  from  the  surrounding  tissue  in  an  actual  surgery,  and  that  there  is  no  reason  for 
the  simulator  to  be  able  to  do  this.  Instead,  the  canaliculus  is  contained  within  the  muscle  and  tissue 
that  makes  us  the  medial  canthal  tendon.  Therefore,  the  simulated  canaliculus  need  only  be  a  channel 
that  is  capable  of  being  located  by  a  whitish  color  and  is  capable  of  being  intubated.  When  lubricated 
with  a  small  amount  of  silicone  oil,  the  part  pictured  below  fulfills  this  requirement. 


Canalicular  channels 


Figure  38,  Canalicular  channels 


Neck  mechanism  design  -  air  muscle  improvements 

During  Q5,  we  began  testing  the  neck  actuation  system,  first  assembled  at  the  end  of  Q4.  Early  in  the 
quarter,  we  performed  initial  testing  of  single  elements  of  the  pneumatic  (McKibben)  muscles  that  will 
generate  neck  motion  and  neck  compliance.  We  then  constructed  a  solenoid  valve  controller  for 
simultaneous  control  of  all  of  the  muscles,  contacted  Dr.  Yong-Lae  Park  at  the  Harvard  Wyss  Institute 
(Harvard  Biorobotics  Laboratory)  for  advice  on  optimizing  the  air  muscle  design  and  developed  a 
controller  to  test  the  motion  capabilities  of  the  current  system. 

Dr.  Park's  advice  led  to  selection  of  an  alternate,  looser  weave  braid  material  that  allows  for  a  larger 
expansion  limit  of  the  muscle  and  thus  a  larger  contraction.  In  addition,  a  closer  fit  between  the  internal 
diameter  of  the  braid  and  the  external  diameter  of  the  inner  pneumatic  tube  prevents  a  buckling 
phenomenon  that  we  observed  in  our  early  testing. 

The  solenoid  valve  assembly,  a  modular  system  that  we  employed  previously  and  a  basic  user  interface, 
is  shown  below. 
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Figure  39:  Neck  motion  test-bed  design,  initial  structure  fabrication,  test  system  with  enhanced  muscle  mesh, 
electrical  control  system,  GUI  for  8-direction  motion  control. 


In  Q6,  we  upgraded  the  pneumatic  valving  system  for  control  the  McKibben  muscles  that  will  drive  neck 
motion  (jerking  motions  to  make  intubation  of  a  combative  patient  more  realistic  and  difficult).  This 
system  makes  use  of  one  3-way  latching  valve  for  each  air  muscle  (supplies  pressure  or  exhausts  the  air 
muscle),  supplied  from  a  common  manifold.  The  air  muscle  exhausts  are  combined  into  a  second 
common  manifold  which  is  in  turn  exhausted  to  atmospheric  pressure  through  a  series  of  relief  valves, 
so  that  air  the  muscles  are  exhausted  quickly  or  slowly  depending  on  which  relief  valve  is  activated. 

As  described  above  in  the  context  of  the  eye  motion  controller,  earlier  tests  made  use  of  an  Arduino 
2009  microcontroller.  With  four  latching  input  valves  and  four  latching  exhaust  valves  in  the  pneumatic 
design,  requiring  two  digital  control  lines  each  for  a  total  of  16  digital  outputs,  we  acquired  an  Arduino 
Mega  microcontroller,  which  has  54  digital  input/output  connections.  The  neck  motion  control  software 
was  modified  to  make  full  use  of  the  Mega's  capabilities,  and  additional  capabilities  added  for 
interfacing  with  the  mannequin. 

With  this  more  capable  system,  we  extended  the  basic  function  tests  performed  earlier  to  perform  the 
same  tests  using  the  mannequin's  power  and  pneumatic  supplies,  crucial  in  creating  the  mannequin- 
integrated  system  that  is  one  of  our  primary  goals. 
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Figure  40:  Pneumatic  control  system  with  fully  enabled  latching  valves  for  four  air  muscles  and  four  exhaust  back¬ 
pressure  valves.  Arduino  Mega  and  control  circuitry  in  prototype  form  on  mannequin's  chest,  valves  and  manifolds 
on  right  shoulder,  pressure  regulator  on  left  shoulder,  test  platform  for  air  muscles  adjacent  to  neck-region. 

Lastly  in  this  section,  we  have  begun  investigations  into  optimizing  the  McKibben  muscles  for  low- 
pressure  supply  air  from  the  mannequin  (rated  at  approximately  1  atmosphere  by  Laerdal),  by  sourcing 
thinner-wall  elastic  tubing  for  the  muscles'  internal  bladders  and  changing  the  fitting  diameters  to 
minimize  the  pinching  that  distorts  the  muscle  end  geometry  at  present.  Additional  details  are 
described  in  later  sections  of  this  report. 


Figure  41:  Air  muscle  images  from  Q5  showing  pinching  by  hose  clamps  at  muscle  ends.  Alternate  design  under 
development. 
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2.3.5. b.  Start  developing  augmented  reality  training  guidance  system  for 
enhanced  eye  trauma  simulator. 

Work  towards  creating  the  AR  surgical  microscope  reached  a  substantial  milestone  during  Q5,  with  the 
completion  of  the  addition  of  a  high  brightness  LCD  monitor  and  optics  into  a  commercial  stereo 
microscope.  We  developed  a  graphical  optics  calculation  tool  using  Solidworks,  which  was  used  to 
determine  proper  lens,  mirror  and  beam  splitter  locations,  optimizing  lens  choice  and  LCD  image  size 
based  on  maximizing  the  view  factor  between  the  LCD  and  the  first  lens  of  the  system  in  light  of 
available  COTS  optical  components. 


Figure  42:  Graphical  calculation  tool  for  AR  microscope  design.  Shows  eyepiece  image  size  at  right,  co-axially 
placed  LCD  panel  image  size  at  left  and  reconfigured,  compact  arrangement  reflecting  the  image  from  an  LCD 
placed  above  the  original  optics  of  the  microscope. 

This  graphical  tool  allowed  us  to  test  LCD  panel  selection  and  provided  dimensional  parameters  for 
designing  the  physical  structure  mounting  the  optics.  The  evolution  of  the  system  over  the  last  quarter 
is  shown  below. 
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Figure  43:  Sequential  development  of  AR  microscope.  COTS  trinocular  microscope  head.  Early  external  display 
test.  Initial  inline  test  of  small  LCD  display.  Integration  of  high  brightness  LCD  during  Q5.  CAD  models  of 
microscope  with  more  compactly  integrated  display  and  optics.  Internal  structure  of  AR  microscope  awaiting 
external  shell  fabrication. 

The  second  last  version  of  the  microscope  was  on  display  for  the  TATRC  visitors  at  MHSRS/ATACCC.  It 
was  found  that  the  heat  generated  by  the  high  brightness  display  softened  the  plastic  supporting 
components,  which  have  been  replaced  with  aluminum  in  certain  positions  and  augmented  by  cooling 
fans  driven  by  the  LCD's  power  supply.  Further  developments  of  the  AR  system  are  described  in  section 
2.3.7. 

Gesture  driven  graphical  UI 

During  Q5  we  made  progress  in  the  gesture-driven  sequencing  of  the  instructional  and  feedback 
materials  presented  to  participants.  Rather  than  requiring  that  a  user  interact  primarily  with  keyboard 
or  mouse  to  advance  the  scenario,  the  various  sensing  elements  provide  the  system  with  state  and 
gesture  information.  For  example,  when  setting  up  the  system,  it  detects  whether  the  proper 
instrument  is  connected,  provides  instructions  for  how  to  do  so  (training  the  user  in  how  to  use  the 
system),  and  advances  automatically  when  that  step  is  complete.  Later  in  the  scenario,  the  system 
updates  the  instructional  display  tracking  the  placement  of  sutures,  for  example,  advancing  when  it 
detects  cutting  corresponding  with  the  trimming  of  sutures  and  replacement  of  the  scissors  in  the 
instrument  tray. 
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Attach  needleholder. 


Figure  44:  Examples  of  gesture  driven  scenario  content.  Simulator  preparation  instructions  driven  by  correct 
sensor/instrument  attachment.  In-scenario  feedback  on  completion  of  series  of  sutures. 

We  developed  the  underlying  code  structure  so  that  each  phase  of  the  scenario  and  each  element  of 
instructional  material  have  specific  actions  that  will  trigger  the  advance  to  the  next  phase.  The  structure 
also  allows  branching  and  return  to  earlier  phases  of  a  scenario,  based  on  the  description  built  into  the 
instructional  element. 

The  gesture  recognition  engine  was  consolidated  into  a  single  library  for  implementation  as  a  DLL 
(Dynamic  Link  Library)  with  corresponding  APIs.  This  last  feature  is  important  in  order  to  export  the 
whole  gesture  recognition  engine  in  another  different  application  and/or  to  implement  and  load  a 
different  one  inside  the  data  analysis  software,  making  possible  the  use  of  the  current  software  for 
different  types  of  surgical  specialties. 

Developer  interface  improvements 

In  Q5,  the  developer  interface,  used  for  data  collection  and  export  of  the  data  to  the  analysis  tools 
continued  to  evolve.  Features  added  were  horizontal  scrolling  through  the  full  data  set  and  indications 
of  the  location  of  the  current  view  is  relative  to  the  entire  scenario,  zoom  functions,  display  of  the 
details  of  manually  entered  event  keystrokes,  ability  to  change  the  rate  of  playback,  from  4x  normal  to 
l/64x  normal  speed,  and  selection  tools  to  export  only  desired  subsets  of  the  acquired  data. 

The  display  of  the  virtual  scene,  showing  graphical  models  of  the  instruments  and  the  mannequin  head 
were  decluttered,  showing  only  connected  instruments  rather  than  the  whole  available  suite. 
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We  added  management  tools  for  the  AVI  video  files  that  correspond  with  the  collected  instrument  data. 
We  created  a  new  software  library  that  can  access  AVI  streams  in  input  and  output,  choose  the  proper 
compressor  in  order  to  save  the  AVI  stream  in  the  best  format  to  optimize  hard  drive  space 
consumption  and  the  CPU  overhead. 

Quantitative  scoring  and  data  analysis 

In  Q5,  the  list  of  quantitative  overall  performance  metrics  included: 

•  Total  Activity  within  ROI 

•  Total  Duration  within  ROI 

•  Longest  Run  within  ROI 

•  Total  Transitions  between  ROIs 

•  ROI  Containment 

•  Adaptive  ROIs  measures 

as  well  as  the  beginning  of  a  method  for  examining  hand  tremor. 

Additional  details  on  the  AR  system  are  covered  in  the  sections  below. 

2.3.6.  “Month  8”  -  Design  of  mandibular/maxillary  trauma  modules,  upper  airway  model 
for  use  of  airway  devices,  new  cricothyroidotomy  portal. 

2. 3. 6. a.  Cricothyroidotomy  module  cut  sensing 

In  Q5,  we  made  advances  over  Year  I's  developments  towards  a  cricothyroidotomy  primary  incision 
detection  sensor.  The  earlier  system,  with  strips  of  conductive  material  that  indicated  an  incision  only 
when  completely  transected,  providing  a  yes/no  signal,  but  not  sufficient  information  for  accurately 
locating  the  cut  length  or  orientation,  was  replaced  with  an  analog  system  similar  in  concept  to  resistive 
touch-screen  technology.  A  3-layer  structure  detects  an  incision  through  it  with  a  conductive  blade, 
when  voltage  from  one  of  the  "drive  layers"  is  connected  to  the  sensing  layer  by  the  conductive  blade 
passing  through  them.  A  series  of  circuits  was  created  to  alternate  rapidly  between  the  x  and  y  direction 
layers  so  that  a  near  continuous  output  of  cut  position  coordinates  can  be  fed  to  the  controller. 
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Figure  45:  Drive  circuit  and  sensor  body  for  cricothyroidotomy  incision  sensor.  Sensor  in  place  on  neck  "skin"  and 
over-molding  assembly  to  encapsulate  the  sensor. 

A  concept  for  a  simplified  version  that  uses  only  two  conductive  layers  and  employs  commercially 
available  neoprene  rubber  with  conductive  surfaces  (previously  used  in  the  original  yes/no  concept)  was 
explored,  however  high  and  variable  contact  resistance  between  the  thin  surface  layers  and  the  incising 
blade  made  this  solution  unworkable  at  present. 

2.3.6.b.  Hemorrhage  control  sensing  system 

Also  during  Q5,  we  have  developed  a  fluidic  sensing  circuit  for  hemorrhage  control  detection. 
Expanding  on  design  concepts  from  our  COMETS  hemorrhage  control  design,  work  was  conducted  on 
the  development  of  a  simple,  rugged,  and  intelligent  system  for  detecting  attempts  at  controlling 
hemorrhage  from  multiple  sources.  Specifically,  we  envision  a  network  of  simulated  vasculature  which 
spans  the  entire  head  and  neck  region  of  our  simulator.  Instead  of  placing  flat  pressure  sensors 
underneath  each  "target  zone"  where  a  medic  may  apply  pressure  to  stop  bleeding,  we  investigated  a 
method  of  detecting  subtle  signatures  of  treatment  in  the  blood  pressure  feedback  loop.  By  using  a 
single,  inline,  pressure  transducer  we  will  be  able  to  detect  and  respond  to  treatment  in  an  intervention 
independent  manner.  It  was  our  experience  with  the  COMETS  system  that  designs  that  rely  on  specific 
pressure  thresholds  to  be  exceeded  tended  to  fail  during  actual  usage.  Therefore,  we  designed  a  system 
that  uses  higher-level  event  observation  to  evaluate  treatment. 

By  monitoring  the  pressure  signal  from  the  inline  transducer,  we  can  observe  changes  in  pressure  of  the 
overall  system  as  well  as  spikes  that  correspond  with  treatment.  Pictured  below,  the  raw  pressure  signal 
(blue)  from  the  sensor  can  be  analyzed  to  extract  the  signatures  (red)  from  applying  or  releasing  direct 
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pressure  on  the  hemorrhaging  vessel.  When  combined  with  the  average  systemic  pressure,  we  should 
be  able  to  more  reliably  detect  treatment  of  non-life  threatening  hemorrhage  controls  in  areas  such  as 
the  superficial  face  and  eye  orbit.  While  these  are  initial  tests,  they  have  resulted  in  promising  data  and 
warrant  continued  exploration. 


Figure  46:  Raw  hemorrhage  back-pressure  measured  by  restricting  flow  through  one  or  more  branches  of  the 
vasculature  (blue).  Derivative  of  pressure  (dP/dt)  for  use  to  flag  when  significant  changes  occur,  e.g.  start  or  end  of 

hemorrhage  control  (red). 

We  tested  the  Deltran  DPT-100  pressure  transducer  from  Utah  Medical  which  is  a  clinical  device  used  on 
actual  patients.  It  is  a  cheap,  5VDC  sensor  that  is  capable  of  monitoring  a  pressure  range  of  -30  mmHg  to 
300  mmHg  without  the  chance  of  cavitation  and  without  require  a  separate  priming  circuit. 

The  signal  from  the  DPT-100  sensor  is  quite  small  and  somewhat  noisy,  so  amplification  and  signal 
filtering  is  required.  We  built  an  initial  prototype  that  allows  the  DPT-100  to  connect  to  an  industry- 
standard  instrument  amplifier  (INA125P)  and  on  to  a  low-cost  Arduino  for  data  collection  and  analysis 
(Figure  47).  A  15Q  resistor  was  used  as  a  gain  resistor  for  the  INA125P,  though  this  will  be  replaced  with 
a  20Q  cermet  trimmer  potentiometer  to  allow  for  more  accurate  gain  control. 


Figure  47:  Test  assembly  for  hemorrhage  control  pressure  sensing.  Clinical  pressure  sensor,  signal  amplifier  chip, 
microcontroller  kit  for  data  acquisition. 
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During  Q6,  discussion  with  Dr.  Troulis  concerned  the  types  of  facial  fractures  that  would  be  most 
relevant  in  the  context  of  facial  trauma  and  complications  that  would  necessitate  surgical  cric.  She 
recommended  that  bilateral  mandibular  fracture  can  allow  the  lower  jaw  to  collapse,  blocking  the  oral 
airway.  We  also  discussed  typical  responses  to  jaw  fracture,  including  wiring  of  the  lower  to  the  upper 
jaw  to  stabilize  the  bones,  which  in  turn  helps  to  reduce  or  stop  ongoing  oral  hemorrhage.  We  will 
explore  options  for  designing  such  features  into  the  mandibular  and  non-ocular  facial  trauma  elements. 

Remaining  developments  in  this  section  remain  for  the  NCE  period  in  Year  3. 

2.3.7.  “Month  10”- 

2. 3. 7. a.  User  testing  of  enhanced  eye  trauma  simulator  at  USUHS  or  Madigan 

Army  Medical  Center. 

As  described  in  the  administrative  sections  above,  we  completed  preparation  of  protocol,  continuing 
review  and  amendments  for  the  study  protocol  and  received  approvals  from  our  IRB  and  ORP  HRPO  for 
a  second  data  collection  exercise  and  participation  in  the  2013  USUHS  Ocular  Trauma  course.  A  new 
vendor  agreement  was  prepared  to  bring  the  simulator  onto  the  USUHS  campus  for  the  testing  exercise. 

During  Q8,  as  with  Year  I's  testing,  we  had  25  lid  laceration  modules  available  for  the  students  and 
faculty  to  use,  all  of  which  were  used  over  the  two-day  wet-lab  sessions  of  the  course.  As  with  Year  1, 
student  availability  was  limited  due  to  the  primary  requirements  for  their  participation  in  the  animal  and 
lecture  sessions.  A  software  error  resulted  in  certain  demographic  data  being  left  out  of  the  database, 
however  we  verified  that  at  least  ten  PGY2  and  PGY3  residents  used  the  simulator.  The  remainder  of 
the  modules  were  used  by  faculty,  of  whom  three  were  oculoplastics  specialists. 

All  participants  filled  out  a  survey  form  which  was  requested  by  Dr.  Marcus  Colyer,  course  director,  to 
provide  feedback  on  participant  impressions  about  the  simulator.  This  survey  (included  in  the  Appendix) 
presented  Likert  scale-type  questions,  asking  for  agreement/disagreement  or  relative  opinions  and,  on 
the  following  topics: 

•  User  interface  and  teaching  content 

o  Teaching  content  was  clear  and  understandable 
o  Interface  guidance  was  helpful  in  learning  the  technique 
o  There  was  too  much  detail  in  the  teaching  content 

•  Eyelid  laceration  module 

o  Tarsal  plate  hardness  was  too  hard/too  soft 
o  Eyelid  hardness  was  too  hard/too  soft 
o  Resistance  to  needle  puncture  was  too  much/too  little 
o  Friction  of  passing  the  needle  through  the  tissue  was  too  much/too  little 
o  Friction  of  passing  suture  through  the  tissue 

•  Simulator  stand  and  instruments 

o  Structure  of  the  stand  interfered  with  interaction  with  the  surgical  field 
o  Cables  and  connectors  interfered  with  performing  the  procedure 
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o  Cable  management  bracelets  and  attachments  interfered  with  performing  the 
procedure 

o  Instruments  were  easy  to  take  from  and  return  to  the  rack 
o  Sutures  were  easy  to  access 
o  Lights  were  in  a  suitable  position 
o  Lighting  brightness  was  too  dim/too  bright 

For  each  section,  and  open  ended  comment/suggestion  section  was  included. 

During  Q8,  we  performed  preliminary  analysis  on  the  data  compiling  the  results  for  the  scored 
questions. 

For  the  interface  and  teaching  content,  the  experts  and  novices  showed  the  same  overall  preferences. 
The  majority  agreed  or  strongly  agreed  that  the  content  was  clear  and  understandable  and  that  it  was 
helpful.  Both  groups  however,  disagreed  that  there  was  too  much  detail  in  the  content.  We  interpret 
this  to  mean  that  there  was  insufficient  detail  and  that  more  should  be  added.  Among  the  comments, 
were  requests  for  additional  detail  generally,  more  specific  instructions  regarding  details  of  the  knots 
and  suture  techniques  to  be  used  in  the  different  phases  of  the  procedure  as  well  as  acceptable 
alternative  methods,  and  information  on  suture  depth  and  tissues  to  approximate. 


Clear  content? 


Helpful  content? 


■  resident 

■  expert 


■  resident 

■  expert 


Too  much  detail? 


■  resident 

■  expert 
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Figure  48:  Simulator  content  responses  between  residents  and  experts 


For  the  eyelid  laceration  module  itself,  experts  and  novices  tended  to  have  similar  responses  as  well. 
Tarsal  plate  hardness  was  judged  almost  evenly  split  between  about  right  or  too  soft;  eyelid  hardness 
was  about  right  or  too  hard  for  most  respondents;  puncture  resistance  was  about  right  for  the  majority 
of  experts  and  residents;  needle  friction  was  about  right  for  the  majority,  and  suture  friction  was  either 
about  right  or  too  much,  evenly  divided  between  experts  and  residents.  Comments  were  fewer, 
however  there  were  requests  for  additional  and  more  obvious  anatomical  landmarks.  These  can  be 
included  through  changes  in  pigment  added  to  the  future  versions  of  the  modules  and  alterations  in  the 
mold  to  include  features  such  as  the  Meibomian  gland  pore,  which  were  present,  but  not  particularly 
deep. 


56 


Tarsus  hardness 


Eyelid  hardness 


■  resident 

■  expert 


■  resident 

■  expert 


nnuch  too  about  too  nnuch 
too  soft  right  hard  too 
soft  hard 


nnuch  too  about  too  nnuch 
too  soft  right  hard  too 
soft  hard 


Puncture  resistance 


Needle  friction 


nnuch  too  about  too  nnuch 
too  nnuch  right  little  too 
nnuch  little 


■  resident 

■  expert 


10 

8 

6 

4 

2 

0 


nnuch  too  about  too  nnuch 
too  nnuch  right  little  too 
nnuch  little 


■  resident 

■  expert 


Suture  friction 
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Figure  49:  Lid  laceration  nnodule  physical  characteristics  responses 

For  the  simulator  stand  and  instruments,  responses  were  in  certain  cases  more  broadly  distributed. 
The  median  responded  that  the  stand  did  not  interfere,  however  there  were  some  who  felt  that  it 
interfered  somewhat  (possible  that  the  agree/disagree  statement  was  misunderstood  and  inverted). 
Cables  and  connectors  clearly  were  troubling  for  some,  but  not  for  others.  The  cable  management 
bracelets  were  roughly  evenly  distributed  between  those  who  were  neutral,  disagreed  or  strongly 
disagreed  that  they  interfered  with  the  task.  The  instrument  access  using  the  tray  and  the  connection 
mechanism  were  found  to  be  easy  to  use  by  nearly  all  participants.  The  lighting  system  was  found  to  be 
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about  right  by  most,  although  too  dim  for  some  users.  The  majority  of  the  comments  concerned  the 
cabling  attached  to  the  instruments,  with  some  requests  for  wireless  instruments  (which  is  not  possible 
given  current  magnetic  tracker  state  of  the  art  technology  and  requirements  on  the  size  and  mass  of  the 
component  that  would  be  attached  to  the  instruments),  others  suggesting  alternate  cable  management 
approaches. 

Discussions  with  Dr.  Colyer  following  the  course  indicated  that  we  would  be  welcome  to  participate  in 
the  2014  course  as  well. 


Stand  interferes? 


Cables/connectors? 


■  resident 

■  expert 


■  resident 

■  expert 


Bracelets  interfere? 


Instm't  access  easy? 


Instm't  connection? 


Light  brightness? 


■  resident 

■  expert 


much  too  about  too  much 
too  much  right  dim  too 
much  dim 


■  resident 

■  expert 


Figure  50:  Standalone  simulator  characteristics  responses. 
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2.3.7.  b.  Pass  upgraded  proof-of-concept  simulator  specifications  onto  commercial 

partner. 

Year  2  was  focused  primarily  on  development  of  the  simulator  systems.  We  maintain  the  confidentiality 
agreement  with  Laerdal  and  were  in  periodic  contact  with  them  regarding  interfacing  our  system  with 
their  mannequin  operating  system,  however  did  not  reapproach  them  for  consideration  of 
commercialization  of  the  system. 

As  described  above,  we  submitted  three  invention  disclosures  to  our  Research  Ventures  and  Licensing 
office  prior  to  the  MMVR  demo,  which  they  converted  into  a  provisional  patent  application.  During  Q8 
a  subsequent  invention  disclosure  was  made  covering  the  system  as  a  whole,  to  be  converted  to  a 
provisional  patent  prior  to  presentation  of  a  poster  at  MHSRS  2013. 

2.3.7. C.  Continue  working  on  augmented  reality  guidance  system  for  enhanced 

eye  trauma  simulator. 


AR  Microscope 

As  described  in  the  sections  above,  we  continued  to  make  progress  towards  developing  the  augmented 
reality  components  of  the  system.  During  Q6,  the  operating  microscope  reached  the  first  useable  state, 
with  the  optics  for  both  left  and  right  eye  implemented,  measurement  of  the  zoom  level  available  and 
power  circuitry  for  the  LCD,  backlight  and  surgical  light  source,  creation  of  software  tools  to  display 
stereoscopic  views  and  mounting  of  an  Ascension  position  tracking  sensor  to  the  microscope  to  track  its 
location  relative  to  the  mannequin  head. 

Following  this  early  work,  the  software  architecture  has  been  modified  to  display  a  model  of  the 
microscope  within  the  virtual  environment  of  the  data  acquisition  interface,  and  the  same  position 
tracking  information  is  used  to  define  the  stereoscopic  imagery  shown  to  the  user  through  the 
microscope.  As  the  microscope  is  translated  or  rotated,  the  view  changes  to  match.  At  this  stage,  test 
patterns  were  displayed;  the  next  steps  in  this  area  include  implementing  views  of  the  virtual 
environment  for  overlay  onto  the  view  of  the  real  objects. 


Figure  51:  AR  microscope  with  enclosure,  internal  LCD,  ring  light  (at  bottom),  position  sensor  mounted  to  right  of 
lower  cowling.  Graphical  model  of  microscope  in  virtual  environment  for  calculation  of  relevant  display  to  be 
presented  on  LCD  and  viewed  through  eyepieces. 
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Figure  52:  Early  test  patterns,  one  image  for  each  eye  piece. 


Figure  53:  Monocular  view  of  combined  image  (left)  and  external  view  of  test  object  of  known  geometry  (center). 
Simulated  rendering  of  apparent  relative  positions  of  virtual  and  real  objects. 

Shown  above  it  can  be  seen  that  the  images  are  rotated  with  respect  to  each  other  (note  colored  rings 
around  target  object).  This  was  necessary  because  the  optical  paths  of  the  eye  pieces  diverge  from  each 
other  by  10  degrees.  To  make  the  LCD  image  appear  aligned  through  the  eyepieces,  the  LCD  image 
must  be  rotated  by  5  degrees  each  in  the  opposite  directions.  Control  panel  elements  were  added  to 
our  primary  data  acquisition  interface  to  allow  for  this  rotation  as  well  as  providing  offsets,  skewing  and 
scaling  to  the  output  images. 
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Figure  54:  Control  panel  for  nnicroscope  view  positioning  and  alignnnent. 


We  observed  that  while  the  images  are  aligned,  the  pixels  grids  viewed  through  the  eyepieces  are  still  at 
angles  with  respect  to  each  other.  Image  quality  would  be  improved  further  if  the  pixel  grids  were 
aligned,  which  required  modification  of  the  optical  path  set  up  between  display  and  eye  piece  lenses. 

The  earlier  2D  graphical  tool  described  above  assumed  that  the  path  from  display,  through  lenses  and 
mirrors  to  the  eyepiece  lay  on  a  plane.  Relaxing  this  assumption  and  allowing  a  3D  path  permits  the 
generation  of  a  path  with  chirality  (rotation).  The  3D  calculation  tool  sets  the  LCD  images  to  be  aligned 
with  the  pixel  array  and  the  output  optical  axis  aligned  with  the  eyepiece.  Four  points  defined  on  the 
LCD  plane  are  projected  through  the  series  of  lenses  and  mirrors  to  their  image  positions.  The  rotation 
of  lines  connecting  pairs  of  image  points  shows  the  rotation  of  the  image.  By  adjusting  mirror  locations 
and  angles,  the  LCD  image  can  be  rotated  optically  (rather  than  graphically),  resulting  in  the  desired 
alignment.  Once  the  mirror  angles  were  calculated,  the  mirror  holding  components  in  the  microscope 
were  modified  to  match,  and  were  installed,  replacing  the  earlier  versions. 


Figure  55:  Front,  isometric  and  side  views  of  new  3D  optical  path  calculator. 


AV  Recording  and  Compression 

After  the  implementation  of  the  different  components  required  for  the  video  acquisition,  in  Q5  all  these 
software  components  were  being  combined  in  order  to  create  the  architecture  to  manage  the  video 
acquisition  (webcam  interface,  that  has  required  a  manual  decoding  YUV  filter  implementation)  a  codec 
manager  (for  the  video  compression  into  the  desired  video  format)  and  the  AVI  manager  (already 
implemented  in  the  previous  months.  Remaining  for  completion  is  synchronization  between  the 
instrument  tracking  data  and  the  video  so  that  the  synthetic  view  playback  corresponds  with  the  video 
playback.  This  will  enable  scrolling  through  recorded  data  from  a  scenario  and  selection  of  sub¬ 
elements  for  more  detailed  review  and  analysis. 
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Figure  56:  Updated  data  acquisition  interface  including  new  video  capture  elements.  Available  CODEC  list  and 
camera  settings  at  left.  Instrument  tracking  data  streams  shown  in  colored  bars  at  bottom 

Leap  Motion  Device 

Our  Year  1  tests  of  the  Kinect  system,  which  we  are  still  evaluating,  suggested  that  the  tracking 
resolution  was  insufficient,  in  part  because  the  minimum  working  distance  between  the  Kinect  and  the 
surgical  field  is  too  large.  A  novel  hand/finger  tracking  device  called  Leap  Motion 
(www.leapmotion.com)  has  recently  appeared  on  the  market,  and  early  release  of  development  systems 
has  been  made  available  to  developers  with  interesting  applications,  as  determined  by  review  of 
abstracts  submitted  to  the  company.  In  Q6,  we  obtained  one  Leap  system  and  have  began  to  evaluate 
its  capabilities.  At  present,  it  is  clear  that  it  can  detect  up  to  10  finger  tips  within  its  working  space,  a 
space  approximately  2A"  on  a  side,  and  it  appears  that  it  is  much  better  suited  for  hand/finger  tracking 
than  the  Kinect.  An  additional  intent  of  using  hand/finger  tracking  is  to  make  the  user  experience  more 
comfortable,  using  the  simulation  platform  for  content  management,  through  the  use  of  the  Leap. 


Figure  57:  Photo  of  Leap  unit,  with  mini-USB  connector.  Product  photo  from  Leap  web  site;  demo  application 
available  with  device  shows  only  line  segments  for  finger  tips,  not  rendered  hands. 

In  Q7,  we  began  to  study  the  possible  uses  of  the  Leap  Motion  finger/hand  tracking  system,  through 
modification  of  the  example  programs  included  with  the  SDK,  such  as  the  Sample_VS2008  project,  which 
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demonstrates  access  to  the  raw  data  coming  from  the  sensor.  The 
device  performs  similarly  to  videos  displayed  on  the  vendor  website,  as 
which  we  confirmed  for  ourselves  through  use  of  the  example 
programs  and  software  demos  that  it  is  possible  to  download  from 
Internet. 

The  sensor  provides  multiple  finger  positions  and  orientations  (each 
tracked  finger  in  five  degrees  of  freedom)  and  the  code  generates  extra 
data  (timestamp,  number  of  fingers,  frame  ID).  Mounted  under  a 
monitor  positioned  to  the  side  of  the  simulation  platform,  the  Leap 
could  track  surgical  instruments  (possibly  out  of  range  of  the  tracking 
system)  or  surgeon's  gloved  fingers  and  provide  indications  on  the 
screen  -  sharp  instruments  and  gloved  fingers  being  incompatible  with 
various  touch  screen  technologies. 

Event  Driven  Gesture  Analysis 

During  Q6,  we  met  with  several  of  our  subject  matter  experts  to 
continue  our  framing  of  the  educational  goals  of  the  upper  eyelid 
laceration  repair  module.  While  very  difficult  to  depict  graphically,  our 
state  diagram  of  the  entire  procedure  grew  extensively.  Based  on  input 
from  the  SMEs,  we  mapped  the  entire  procedure  from  start  to  finish, 
including  many  actions  we  do  not  intent  to  simulate  with  this  system. 
The  purpose  of  this  exercise  being  to  understand  what  context  the  user 
is  being  placed  in  when  using  the  simulator.  States  of  the  procedure 
such  as  requesting  an  x-ray  of  the  patient  would  be  carried  out  in 
software  as  a  series  of  interactive  steps  in  the  GUI  rather  than  an 
interaction  with  the  simulator  mannequin. 

Several  areas  of  the  training  module,  when  mapped,  started  to 
illustrate  repetitive  functions.  We  have  captured  these  repetitive 
functions  (actions,  gestures,  thresholds,  etc.)  and  are  using  them  to 
drive  our  software  development  processes.  For  example,  our  ability  to 
detect  a  specific  sequence  of  events  associated  with  placing  the  needle 
through  the  eyelid  and  retrieving  it  through  the  other  side  forms  a 
basic  building  block  in  our  suturing  gesture  recognition  system.  If  the 
system  detects  a  second  pattern  of  this  type,  we  know  that  the  user 
has  taken  two  bites  to  pass  the  suture  through  the  location  in  the 
wound.  Should  these  events  be  followed  by  a  knot  tying  gesture,  we 
are  then  able  to  warn  the  user  that  it  is  advisable  to  leave  the  sutures 
untied  until  all  of  the  sutures  have  been  placed  and  the  lid  margin 
approximated.  This  is  necessary  for  both  real-time,  in-context  surgical 
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guidance  for  novices  as  well  as  forming  the  foundation  of  our  gesture  recognition  framework. 

As  of  Q6,  the  description  hierarchy  tree  had  10  levels  of  description: 

•  Curriculum:  1  entry  (eye  trauma  course) 

•  Module:  1  entry  (marginal  lid  laceration) 

•  Styles:  1  entry  (''textbook''  sequence  -  later  entries  to  include  surgeon-specific  techniques) 

•  Steps:  5  entries  for  current  style  (e.g.  examination,  suturing  tarsal  plate,  incision  closure)  - 
different  numbers  of  Steps  will  apply  to  alternate  Styles 

•  Tasks:  52  defined  across  all  Steps,  with  varying  numbers  for  each  Step  -  marginal  suture  through 
tarsal  plate 

•  Gestures:  may  be  simple  (review  X-ray)  or  compound,  with  further  divisions  (e.g.  2-1-1  suture 
gesture) 

•  SubO-Gesture:  optional  -structure  for  capturing  needle  passages  without  knot  tying 

•  Subl-Gesture:  high  level  components  of  gestures,  e.g.  single  bite  suture,  2-throw  knot,  1-throw 
knot,  cut  suture 

•  Sub2-Gesture:  mid  level  gesture  component,  e.g.  tissue  grasping,  needle  bites,  repetitions  of 
throws,  suture  grasping,  knot  tensioning 

•  Event:  sensor-driven  events  that  comprise  the  Sub2-Gestures:  instrument  jaw  opening/closing, 
applied  force  level  threshold  transitions,  passage  across  region  of  interest  boundaries,  focused 
path  analysis 

Sample  elements  of  the  hierarchy  are  depicted  in  the  sidebar  above. 

Simulation  Content  Frame  Structure 

Extending  the  content  frames  described  earlier,  during  Q7,  the  scripting  language  (XML-based) 
developed  to  manage  the  simulation  content  was  been  extended,  introducing  new  functionalities  and 
extending  the  syntax. 

The  new  frame  (underlying  description  of  the  content  and  functionality  for  a  given  node  in  the 
procedure)  in  the  scripting  language  allows  the  positioning  of  the  slide  buttons  and  extends  the  type  of 
buttons  present  in  the  simulation  flow. 

To  support  good  coding  practice  and  simplify  development  for  future  users,  the  capability  to  add 
comments  in  the  frame  structure  has  been  added.  Further  replacement  of  the  system  condition  index 
(numbers  between  the  <system_Cond>  tags)  with  a  set  of  mnemonic  labels  is  being  implemented.  A 
sample  of  the  most  recent  version  is  shown  below. 

The  source  code  was  prepared  to  export  the  gesture  recognition  core  into  an  external  DLL.  The  ongoing 
work  has  been  developed  based  on  the  specifications  defined  previously  several  months  ago.  The  code 
has  also  been  commented  in  order  to  make  it  more  accessible  to  other  members  of  the  development 
team  and  it  is  still  going  ahead  a  cleaning  of  the  entire  software  project. 
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In  order  to  support  the  externalization  of  the  gesture  recognition  core  has  been  necessary  to  insulate 
the  shared  structures  and  library  between  the  main  program  and  the  DLL,  creating  a  common  folder  and 
moving  inside  specific  shared  files  data  structures,  in  order  to  keep  the  code  consistent  in  both  projects 
after  modifications  on  the  shared  structures. 

The  changes  have  been  made  on  the  libraries  that  manage  the  simulation  content  slides  and  the  event 
handlers  used  in  between  them.  The  different  event  handlers  have  been  exported  into  the  DLL  library 
and  the  importing  interface  is  about  to  be  completed. 

The  scripting  language  used  to  code  the  simulation  content  (slide  set)  was  extended  incorporating  the 
name  of  the  proper  DLL  to  use  in  order  to  handle  the  right  events  for  the  specific  simulation. 

Another  new  feature  that  is  almost  ready  is  the  introduction  of  separate  warning  handlers  that  are 
currently  coded  in  the  same  channels  used  for  the  event  handlers,  extending  to  5  the  total  number  of 
event  handling  channels  for  each  slide. 


//  first  slide  with  menu  selection 

<Section> 

<ID>1</ID> 

<Slide>Contents\Simulations\MMVR\Slides\2013MMVROCFGUI . 001 . jpg</Slide> 
<Audio>none</Audio> 

<Light>0</Light>  //  no  light  on  the  mannequin 

<Events> 

<Buttonl>0, 1, 0 . 0000, 0 . 4870</Buttonl>  //  NO  Button  on  the  left  side  of  the  screen 
<Button2>3, 23, 0 . 8339, 0 . 4870</Button2>  //  Next  Button  on  the  right  side  of  the  screen 

<Button3>0, 0,0.4169,0. 9000</Button3> 

<Time>0, 2</Time> 

<system_Cond>-l , 0</ system_Cond> 

<system_Cond>0 , 0</ system_Cond> 

<system_Cond>0 , 0</system_Cond> 

<system_Cond>0 , 0</system_Cond> 

<system_Cond>0 , 0</system_Cond> 

<Warning_Cond>0 , 0</Warning_Cond> 

<Warning_Cond>0 , 0</Warning_Cond> 

<Warning_Cond>0 , 0</Warning_Cond> 

</Events> 

</Section> 


The  architecture  has  also  been  modified  splitting  in  two  different  sections  the  content  slides 
management  and  the  gesture  segmentation.  These  two  last  components  now  run  independently. 

Some  of  the  Gesture  recognition  routines  have  been  improved  in  order  to  make  them  more  accurate 
and  more  tolerant  to  wrong  movements  into  the  workspace. 

Some  other  time  has  been  spent  merging  the  last  version  of  the  source  code,  developed  during  the 
MMVR  conference  with  the  latest  developments  of  the  one  on  the  workstation. 
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Gesture  Detection  Algorithm  Development 

The  relatively  simple  early  approach  we  developed  to  drive  progression  through  the  scenario  content  is 
being  expanded  with  a  richer  set  of  algorithms  have  been  created  in  order  to  recognize  the  different 
gestures  performed  by  the  surgeon  during  the  procedure.  This  is  based  on  a  more  extensive  set  of 
elemental  states,  listed  below.  This  list  is  extended  with  an  initial  set  of  error  conditions  that  will  trigger 
warnings  from  audio  cues  to  video  instructions  on  correct  technique. 


ID 

Routine  Description 

-2 

Stop  state  machine 

-1 

restart  state  machine 

0 

NULL 

1 

Scissor  is  cutting  inside  a  specified  ROI 

2 

true  if  a  forceps  is  connected 

3 

true  if  no  forceps  are  connected 

4 

True  if  a  needle  holder  is  connected 

5 

True  if  no  needle  older  are  connected 

6 

True  if  an  instrument  is  inside  a  specific  ROI 

7 

True  if  a  scissor  enter  inside  a  specific  ROI,  cut  and  after  goes  out 

8 

Record  data  for  scoring 

9 

Plot  scoring  results 

10 

True  if  the  spear  is  connected 

11 

True  if  the  spear  enter  in  a  specific  ROI  and  remains  there  for  at  least  3 
seconds 

12 

True  if  the  forceps  enters  into  a  specific  ROI  and  remains  there  for  at  least  3 
seconds 

13 

True  if  a  double  pass  suture  is  performed  (first  pass) 

14 

True  if  a  double  pass  suture  is  performed  (second  Pass) 

15 

True  if  2-1-1  is  performed  (not  complete) 

ID 

WARNING  CHECKING 

100 

Check  for  too  hard  forceps  grasping  (A) 

101 

Check  for  too  hard  forceps  grasping  (B) 

130 

Check  for  single  pass  suture  instead  of  a  double  one  (A) 

131 

Check  for  single  pass  suture  instead  of  a  double  one  (B) 

We  are  investigating  possible  extensions  for  the  scripting  language,  namely  including  a  numerical 
parameter  for  certain  routines,  in  order  to  create  different  settings  (e.g.  trim  length  of  sutures)  allowing 
us  to  use  the  same  algorithm  but  with  different  settings  for  different  types  of  users. 

A  series  of  schematics  of  the  state  machine  structure  being  implemented  in  code  is  shown  below,  now 
with  the  various  links  between  states  defined  by  the  test  functions  that  determine  which  threshold  has 
been  crossed. 
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Whole  procedure  overview  scheme 


(6  o 


lOOKFOR  HFMDRRAGf  m^chms 


Figure  59:  schematics  of  elements  of  state  connections  for  different  levels  of  gesture  recognition  heirarchy 
Content  Development  for  Scenarios 

During  Q7,  the  didactic  content  for  the  eyelid  laceration  module  was  fully-developed  to  incorporate  the 
complete  set  of  steps  involved  from  a  clinical  point  of  view.  These  steps  were  established  after 
reviewing  existing  training  materials  as  well  as  from  several  meetings  with  our  subject  matter  experts. 
While  comprehensive  and  implemented  on  the  simulator,  we  have  not  activated  all  of  the  steps  in  the 
setup  for  use  at  the  2013  USUHS  Ocular  Trauma  Course.  Rather,  we  have  activated  the  steps  that  allow 
a  user  to  be  guided  by  an  instructor  and  immediately  begin  suturing. 
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Figure  60:  full  sequence  of  available  content  and  sequential  guidance  slides  for  lid  laceration  scenario. 

Content,  such  as  that  pictured  below,  for  steps  that  would  typically  be  involved  in  patient  prep  and  early 
diagnosis  are  currently  set  to  ''hidden.''  There  are  currently  61  screens  implemented  in  our  eyelid 
laceration  training  scenario. 


Determine  if  okay  to 
add  eyedrops 


n  MASSACHUSETTS 
^  GENERAL  HOSPITAL 

[PLAY  AUDIO  FILE]  Click  next  button  on  right. 

Figure  61:  Pre-scenario  preparation  informational  slide,  available  for  full  content  use  (hidden  for  2013  Ocular 
Trauma  Course  use) 

To  better  integrate  with  our  underlying  learning  architecture,  the  eyelid  laceration  didactic  content  was 
organized  as  an  individual  module.  There  will  be  several  modules  developed  for  this  simulator,  providing 
an  architecture  that  supports  future  content  is  important.  The  user  begins  by  selecting  the  module 
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he/she  wants  to  train  on  and  clicks  the  start  button  in  the  center  of  the  screen.  Additional  modules 
would  be  found  by  navigating  to  the  left  or  right  of  the  central  area  on  the  screen. 


tMuCLEA 


in  w.\:  ;•>'  -*-• 

(PLAY  AUDIO  KILE]  Ability  to  scroU  through  modules  horizontally  using  arrows  on  left 
and  right  Click  start  to  continue. 

Figure  62:  Scenario  selection  interface  screen 

Once  a  module  is  selected,  the  user  is  then  presented  with  the  ability  to  practice  a  specific  technique  to 
suturing  the  eyelid  closed.  We  found  that  each  surgeon  and  subject  matter  expert  we  consulted  had 
strong  opinions  about  the  ''right''  way  to  provide  guidance  to  the  novice  trainee.  As  they  all  referenced  a 
"textbook"  way  of  suturing  as  an  underlying  foundation  to  their  personal  approach,  we  began  by 
implementing  that  basic  version. 


5lart 


r  -  -  unique 


Start 


[PLAY  AUDIO  FILE]  Ability  to  scroll  through  modules  horizontally  using  arrows  on  left 
and  right.  Click  start  to  continue. 


Figure  63:  Selection  among  different  techniques  for  given  procedure 

Our  design  allows  a  novice  to  learn  the  fundamental  manner  of  suturing  a  laceration  to  the  upper  eyelid. 
Once  completed,  the  user  is  then  able  to  practice  the  individual  techniques  perfected  by  different 
surgeons.  As  they  progress  through  the  scenario,  video  and  audio  commentary  accompany  a  custom- 
tailored  set  of  didactic  content  in  the  interface  to  illustrate  to  the  novice  where  they  are  deviating  from 
the  textbook  procedure  and  why. 
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2.3.8.  “Month  11”  -  Completion  of  prototype  stand-alone  ocular  and  craniofacial  trauma 
training  module,  control  software  supporting  sensing  and  response  functionality. 

This  phase  remains  in  progress.  Description  of  progress  towards  completion  are  reported  primarily 
in  2.3.4,  the  middle  sections  of  2.3.5  and  2.3.7a  and  2.3.7c.  Additional  trauma  module  scenarios  and 
content  are  slated  for  inclusion  in  the  overall  system. 


2.3.9.  “Month  12”  -  Testing  of  fit,  function  of  sensors,  actuators,  replaceable 
components. 


Air  muscle  and  neck  motion  testing 

Following  early  experiments  to  characterize  our  air  muscle  designs,  described  above,  during  Q5,  we 
completed  the  test  rig  to  allow  testing  of  motion  in  all  directions  and  reevaluate  the  capabilities  of  the 
new  muscles  (Figure  64).  This  system  appears  to  be  sufficient  to  generate  neck  motions  that  would  be 
used  in  the  mannequin-integrated  version  of  the  head  (Phase  III)  to  simulate  a  struggling/gagging 
casualty,  a  feature  that  does  not  appear  to  be  present  in  the  commercial  market. 


Figure  64:  Motion  control  of  neck  using  combinations  of  single  or  adjacent  air  muscles.  Activation  of  single  air 
muscle  to  illustrate  range  of  motion.  Base  of  "neck"  at  the  bottom,  "head"  would  be  at  the  top. 

Q6  and  Q7  saw  a  hiatus  in  the  neck  development  work,  with  changes  in  personnel. 

In  Q8  we  continued  this  work  in  designing  a  more  anatomically  correct  neck  structure  that  integrates 
the  neck  pitching  motions  with  head  yaw.  The  mannequin-integrated  version  of  the  system  will  have 
the  active  motions,  while  the  stand-alone  version  may  make  use  of  similar  linkages  to  allow  the  surgeon 
to  passively  move  the  head  with  the  same,  normal  range  of  neck  motion,  but  without  the  actuation 
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components,  thereby  minimizing  metallic  elements  that  may  distort  the  surgical  instrument  tracking 
system. 

Using  the  anatomical  models  that  were  segmented  during  Year  1  and  concepts  developed  during  our 
earlier  COMETS  autonomous  casualty  simulation  mannequin  project,  we  created  a  flexible  neck 
structure  using  a  series  of  vertebral  bodies  and  disks,  each  of  which  accommodates  a  fraction  of  the  full 
flexing  range  of  motion  of  the  human  neck. 


Figure  65:  Anatomically  inspired  neck  structure  CAD  model,  physical  implementation,  and  mounting  to  trauma 
mannequin. 

The  vertebral  bodies  have  flanges  extending  away  from  their  cores  that  mate  with  the  McKibben 
muscles,  so  that  they  rotate  with  respect  to  each  other  when  the  head  turns.  A  central  flexible 
transmission  shaft  provides  tensile  strength  and  links  the  base  of  the  neck  to  a  head  rotation  motor  that 
will  be  mounted  in  the  head.  Human  neck  range  of  motion  is  different  in  flexion/extension  and  lateral 
abduction,  so  the  vertebral  bodies  are  designed  such  that  their  surface  contours  allow  for  more  motion 
before  one  body  collides  with  the  next  at  the  front  and  back  of  the  body  than  the  left/right  direction. 
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Figure  66:  Vertebral  body  and  disk  designs 


In  the  course  of  designing  the  neck  mechanism,  we  reviewed  the  McKibben  muscle  design.  The  earlier 
work  and  inquiry  showed  that  sparse  mesh  permits  larger  overall  deformation.  Thinner  bladder  walls, 
for  a  given  material  stiffness  allow  for  greater  deformation  for  a  given  supply  pressure,  as  does  a  softer 
material  for  a  given  thickness.  We  identified  both  thinner  wall  silicone  tubing  and  lower  durometer 
rubing  and  redesigned  the  muscles  to  accommodate  the  new  material.  In  addition,  we  found  that  the 
original  tube  fittings  used  as  attachment  points,  which  were  smaller  in  diameter  than  the  silicone  and 
mesh  tubing  diameters  resulted  in  pinching  at  the  muscle  ends,  reducing  the  available  amount  of  muscle 
contraction.  Related  to  that,  standard  hose  clamps  that  had  been  used  were  prone  to  pinch  and 
puncture  the  silicone  tubing  walls.  We  replaced  the  fittings  and  hose  clamps  with  larger  diameter 
fittings  and  plastic  snap-grip  hose  clamps  without  the  hard,  sharp  edges. 


With  these  design  changes,  we  reexamined  McKibben  muscle  performance,  comparing  the  thin  and  two 
grades  of  soft  rubber  to  determine  the  best  available  of  materials  sourced  from  industrial  supply  houses. 
The  thin  wall  silicone  was  found  to  have  the  best  performance  of  the  materials  tested. 


Figure  67:  Retesting  McKibben  muscles  with  new  materials,  fittings.  500g  load  results  shown:  relative  contraction 
for  black  latex  (blue),  amber  latex  (red)  and  thin  wall  silicone  tubing  (green) 

A  mating  platform  (described  below)  was  designed  to  attach  the  neck  design  to  the  Laerdal  SimMan 
Essential  mannequin  that  we  acquired  in  Q6.  Using  the  pneumatic  control  system  described  above,  we 
tested  the  range  of  neck  motion.  While  the  neck  assembly  is  capable  of  the  full  human  range  of  motion, 
when  coupled  to  the  air  muscles,  the  available  contraction  appears  to  reduce  the  actual  range  delivered. 
In  addition  while  the  neck  flexes  quite  far  when  the  muscles  contract,  the  platform  shown  (representing 
the  mounting  structure  of  the  skull)  may  pitch  by  a  lesser  amount.  Thus,  at  this  stage,  the  mannequin 
will  move  less  under  its  own  control  than  can  be  generated  by  manual  manipulation  of  the  head/neck  by 
a  trainee.  As  we  progress,  we  will  endeavor  to  increase  the  controllable  motion,  however  for  the 
purposes  of  creating  a  system  that  can  present  autonomous  motion  that  will  indicate  consciousness  and 
present  a  simulation  of  a  casualty  resisting  treatment  (e.g.  intubation/cric),  this  system  will  satisfy  the 
basic  goals. 
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Figure  68:lnitial  range  of  motion  of  neck  actuation  system 


Figure  69:  Manual  manipulation  of  neck  with  attached  skull  model 
Blink  actuation  testing 

In  Q7,  much  progress  was  made  on  the  blinking  mechanism.  Based  on  developments  described  above,  a 
prototype  blinking  portal  was  created  which  demonstrated  an  effective  solution  using  embedded  nylon 
string  in  the  tarsal  plate  which  can  then  be  actuated  by  a  servo.  The  string  can  be  physically  attached  to 
the  substrate  in  future  iterations.  This  solution  is  consistent  with  the  earlier  developments  of  the  servo- 
driven  blink  mechanism  described  in  previous  reports. 

Changes  to  the  portal  were  also  necessary  and  consisted  of  incising  relief  flaps  into  the  silicone.  These 
flaps  allow  the  silicone  to  bend  more  freely  and  control  the  way  in  which  it  deforms.  In  production, 
these  flaps  will  be  incorporated  into  the  mold  design.  This  feature  addresses  one  of  Dr.  Mazzoli's 
principal  suggestions  regarding  the  eyelids,  namely  that  redundant  tissue  should  be  present,  permitting 
more  realistic  manipulation  and  proper  creasing  when  the  eyelid  blinks. 
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Figure  70:  Servo  control  of  module  eyelid  blinking 

2.3.10.  “Month  14”  -  Design  of  modifications  to  trauma  mannequin  for  integration  of 
ocular  and  craniofacial  simulator  structure  module. 

During  Q5  we  made  a  final  inquiry  with  CAE  Healthcare  regarding  their  Caesar  mannequin,  which 
previously  was  not  expected  to  be  available  for  purchase  during  our  timeline.  At  the  MHSRS  2012 
conference,  one  of  the  mannequins  was  on  display  and  the  sales  staff  conveyed  that  they  were  available 
for  distribution.  Examination  of  the  system,  however,  suggests  that  integration  of  a  new  head  onto  the 
mannequin  would  be  more  difficult  than  the  options  we  had  been  examining  with  Laerdal.  In  addition, 
the  Laerdal  system  that  has  sufficient  capabilities  for  our  purposes  costs  approximately  half  as  much  as 
Caesar,  and  further,  the  programming  interface  mentioned  in  Year  1  reports  was  not  available  for 
Caesar.  As  a  result,  we  submitted  a  purchase  requisition  for  a  Laerdal  SimMan  Essential,  which  arrived 
during  Q6. 
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Figure  71:  Disassembly  and  examination  of  Laerdal  SimMan  Essential  mannequin  in  support  of  designing  interface 
components  to  mate  the  new  head/neck  to  the  commercial  body.  Aligned  CAD  models  in  use  for  design  of 
attachments  between  new  head/neck  and  stock  mannequin. 

In  Q6,  the  early  work  involved  opening  the  mannequin  to  identify  power  and  air  pressure  supplies,  test 
its  normal  functionality  under  control  of  the  Laerdal  software,  and  begin  to  look  into  the  SDK  available 
for  programming  the  mannequin  beyond  the  compiled  software  provided.  This  lead  to  a  series  of 
technical  queries  that  were  provided  to  Laerdal. 

Following  our  discussions  with  Laerdal,  they  provided  us  with  limited  technical  details  of  the 
mannequin's  structure,  pneumatic  supply  and  electrical  and  communications  capabilities. 

We  combined  the  CAD  models  of  the  main  internal  structural  plate  from  Laerdal  with  a  3D  laser  scan  of 
the  outer  surfaces  of  the  mannequin  shoulders,  using  a  long  focal  distance  photo  of  the  mannequin  to 
aid  in  aligning  the  head/neck  anatomy  with  the  original  mannequin  head.  This  is  serving  as  the  basis  for 
our  structural  design  of  the  connection  and  mounting  of  our  head  to  the  mannequin's  body. 

As  described  earlier,  we  have  begun  to  use  the  mannequin's  air  and  power  supply  to  drive  the 
mechanisms  for  the  head  and  neck.  Immediately  inside  the  mannequin's  left  shoulder  are  a  series  of 
electrical  connectors,  one  of  which  is  dedicated  for  head  control.  It  includes  terminals  for  12V  supply 
and  a  serial  communications  port  that  commands  the  mannequin's  normal  head.  We  have  added  our 
own  connector  in  place  of  the  original  to  access  power  and  detect  the  pulse-control  signals  of  the 
mannequin,  so  that  our  later  hemorrhage  system  can  be  synchronized  with  the  mannequin's  pulse. 
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The  12V  power  is  now  supplying  the  Arduino  Mega  that  was  mentioned  earlier  and  the  series  of  8 
latching  valves.  As  described  earlier  in  this  report,  in  addition  to  the  control  of  the  valves,  the  Arduino's 
software  has  been  upgraded  to  control  of  up  to  12  servos  (8  are  currently  designed  into  the  blinking/eye 
motion  mechanism),  two  electronically  controlled  pressure  regulators,  up  to  10  PWM-controlled  devices 
(e.g.  fluid  or  air  pumps)  or  other  digitally  controlled  devices  (e.g.  additional  valves  for  hemorrhage),  up 
to  two  stepper  motors  (possible  for  jaw  control,  head  rotation,  measurement  of  16  analog  sensors. 

The  pneumatic  supply  provides  air  pressure  that  varies  from  a  peak  of  approximately  1  atmosphere  and 
falls  by  approximately  25%  before  the  mannequin's  compressor  turns  on  to  restore  pressure.  This 
pressure  drop  is  related  to  the  mannequin's  own  breathing  mechanism,  which  consumes  air  during  each 
breath  (small  internal  bladders  push  the  main  chest  plate  outwards),  allowing  the  system  pressure  to  fall 
since  there  is  no  regulated  internal  reservoir. 

Our  testing  of  the  neck  mechanism  under  this  air  supply  shows  that  with  the  fall  of  system  pressure, 
actuated  air  muscles  lose  force  cyclically  as  the  mannequin  compressor  cycles,  with  slowly  changing 
neck  position  as  pressure  falls,  and  a  return  to  the  set  position  when  the  compressor  runs.  We  are 
evaluating  whether  to  add  a  separate  reservoir  that  maintains  the  neck's  position,  or  whether  this 
motion  provides  the  appearance  of  random  motion  beyond  the  deliberate  actions  controlled  by  our 
valving  system. 


Figure  72:  Air  muscle  system  integrated  with  mannequin  air  and  power  supply.  Blue  tubing  is  connected  in  place 
of  stock  head  air  connector,  twisted  pair  (black/red)  wires  supply  mannequin  power  to  the  solenoid  valve  control 
circuitry  and  the  Arduino  Mega  microcontroller. 
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During  Q7,  we  isolated  one  of  the  problems  that  appeared  during  discrete  component  testing:  while  the 
air  muscle  valves  are  specified  for  use  with  12V  control  signals,  and  the  mannequin's  supply  voltage  is 
also  12V,  the  transistors  that  switch  the  valves  on  and  off  have  their  own  voltage  drops  in  series  with  the 
valves.  As  a  result,  the  voltage  available  to  the  valves  is  reduced  below  12V,  by  an  amount  that  makes 
valve  activation  unreliable.  Two  solutions  were  identified  as  of  the  preparation  of  the  previous  technical 
report:  replace  the  12V  valves  with  their  6V  equivalents,  or  use  a  voltage  converter  which  boosts  the 
12V  supply  up  to  15V  (at  the  cost  of  drawing  more  current  from  the  batteries),  so  after  the  transistor 
voltage  drop,  the  valves  still  have  enough  driving  force  to  reliably  activate.  A  suitable  voltage  converter 
was  ordered,  received  and  tested  to  confirm  that  it  functioned  as  planned.  The  code  modification  to 
properly  sequence  high  current-draw  components  such  as  the  valves  is  in  progress. 

During  Q8  in  conjunction  with  the  neck  mechanism  developments  described  above,  we  used  the  CAD 
models  provided  by  Laerdal  for  the  main  frame  of  the  mannequin  to  create  a  mounting  frame  that 
mates  the  neck  to  the  mannequin  main  body.  The  attached  neck  is  shown  in  figures  earlier  in  this 
section. 


2.3. 11 .  “Month  16”  -  Revision  of  physiology  model  software. 

This  work  began  substantially  during  Q7.  In  the  first  two  months  of  the  quarter,  we  began  to  examine 
the  Laerdal  SDK  in  detail,  which  provides  a  programming  interface  to  the  mannequin's  control  software, 
starting  with  familiarization  with  the  programming  language  (C#)  and  with  the  layout  of  the  VS  (Visual 
Studio)  C#  project.  To  avoid  potentially  damaging  the  mannequin,  when  necessary  we  are  doing 
preliminary  testing  of  code  using  the  Laerdal  virtual  mannequin  system,  which  presents  a  system  that 
responds  to  code  written  using  the  SDK  the  same  way  the  real  SimMan  would. 

The  initial  focus  has  been  the  LaerdalSimulatorAPI.dll  dynamically  linked  library.  The  goal  of  the  work 
was  to  find  out  a  proper  way  to  use  the  functions  and  methods  provided  by  the  dynamic  library,  in  order 
to  use  them  in  a  new  C++  application.  As  the  source  code  and  documentation  of  the  DLL  is  not  provided, 
extrapolation  of  the  content  of  the  DLL  is  necessary.  To  help  with  that  we  used  the  "Object  Browser"  VS 
functionality  and  the  free  .NET  decompiler  and  assembly  browser  "JetBrains  dotPeek  1.0" 

Early  approaches  aimed  at  minimizing  the  number  of  additional  files  needed  and  retaining  as  much 
simplicity  as  possible  in  converting  the  functionality  available  in  the  SDK,  written  in  C#,  to  functions 
compatible  with  our  simulator,  written  in  C++  were  unsuccessful.  A  subsequent  approach  treating  each 
component  of  the  C#  SDK  separately,  creating  a  library  of  files  to  form  a  bridge  between  the  two 
languages  was  attempted  by  one  of  our  interns  next. 

Developing  the  Wrapper  for  LaerdalSimulator.dll 

The  DLL  wrapper  works  as  a  bridge,  linking  the  C#  DLL  with  the  C++  client  application.  In  order  to  create 
the  linking,  every  member  (classes,  delegates,  interfaces  and  enums)  of  the  C#  DLL,  have  to  be  re¬ 
declared  inside  the  wrapper. 


^  Download  and  documentation  available  at  www.jetbrains.com/decompiler 
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To  create  the  wrapper  we  used  an  approach  based  on  creating  a  different  C++  file  for  every  class 
implemented  in  the  original  DLL,  and  another  single  C++  file  which  contains  enumerators  and  interface 
classes  .  Because  of  that,  the  wrapper  project  consists  in  the  following  files: 

-  ExceptionData_Wrapper.h,  which  contains  the  declaration  of  the  class  ExceptionData_Wrapper 
and  its  methods/constructors/destructors 

-  ExceptionData_Wrapper.cpp  which  contains  the  implementation  of  the  methods  of  the  class 
ExceptionData_Wrapper 

-  ManikinException_Wrapper.h  which  contains  the  declaration  of  the  class 
ManikinException_Wrapper  and  its  methods/constructors/destructors 

-  ManikinException_Wrapper.cpp  which  contains  the  implementation  of  the  methods  of  the 
class  ManikinException_Wrapper 

-  ManikinFactory_Wrapper.h  which  contains  the  declaration  of  the  class 
ManikinFactory_Wrapper  and  its  methods/constructors/destructors 

-  ManikinFactory_Wrapper.cpp  which  contains  the  implementation  of  the  methods  of  the  class 
ManikinFactory_Wrapper 

-  LaerdalSimulatorAPI_Wrapper.h  which  contains  the  declaration  of  interface  classes,  delegates 
and  enumerators. 

-  LaerdalSimulatorAPI_Wrapper.cpp  which  contains  the  declaration  of  the  enums  (as  they  are 
declare  already  In  the  .hfile  maybe  I  would  delete  them  from  here)  and  would  contain  (still  to  be 
done)  the  implementation  for  some  get/set  methods  for  the  interfaces. 

-  LaerdalSimulatorAPl.h  header  file  of  the  original  DLL  created  by  me  (need  some  changes). 

To  test  the  wrapper  our  CS  intern  created  a  small  Dialog  application  that  shows  the  client  a  window  and 
performs  a  series  of  functions  based  on  which  dialog  button  is  pressed. 

The  first  test  performed  was  on  the  class  ExceptionData_Wrapper.  It  contains  a  series  of  properties 
(data)  that  can  be  written  and  read.  The  test  was  very  simple  and  did  not  need  require  a  link  to  the 
virtual  mannequin.  The  testing  procedure  was  basically  made  of  the  following  steps: 

•  calling  the  set  method  of  the  property  (the  one  declare  inside  the  wrapper).  This  is  linked  to  the 
set  method  of  the  C#  DLL,  so  whenever  a  new  value  is  passed  to  the  wrapper  set  function,  the 
same  value  would  be  passed  to  the  property  of  the  LaerdalSimulator.dll. 

•  calling  the  get  method  declare  inside  the  wrapper.  Because  of  the  linkage,  this  would  go  reading 
the  value  of  the  property  contained  in  the  C#  DLL 

•  if  the  two  value  are  the  same,  the  linkage  is  verified 

The  test  has  demonstrated  that  a  link  was  built,  a  first  step  towards  establishing  communication 
between  our  application  and  the  mannequin. 

Work  on  this  element  remained  static  during  Q8  with  a  shift  in  emphasis  towards  assigning  analysis  of 
the  USUHS  data  to  the  CS  intern.  It  will  continue  during  the  extension  period  in  Year  3. 
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2.3.12.  “Month  18”- 


2.3. 12.  a.  Completion  of  prototype  integrated  ocular  and  craniofacial  trauma  system 
with  mannequin. 

Efforts  towards  this  milestone  has  been  commenced,  as  described  above. 

2.3.12. b.  Completion  of  augmented  reality  system  supporting  stand-alone 
simulator. 

Efforts  towards  this  milestone  has  been  commenced,  as  described  above. 

2.3.13.  “Month  20”  -  User  testing  of  combined  system. 

Efforts  towards  this  milestone  has  not  been  commenced  at  this  time 

2.3.14.  “Month  23”  -  Completion  of  revisions  based  on  user  testing. 

Efforts  towards  this  milestone  has  not  been  commenced  at  this  time 

2.3.15.  “Month  24”  -  Final  report  submission. 

Efforts  towards  this  milestone  has  not  been  commenced  at  this  time 
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3.  Key  Research  Accomplishments 

•  Conversion  of  proof-of-principle  demo  system  to  final  standalone  eye  trauma  simulator 

o  Design  and  fabrication  of  final  standalone  simulator  main  structure,  including  Kinect, 
projector,  webcam  and  magnetic  position  tracker  integration.  Only  minor  revisions  are 
anticipated 

o  Redesign  of  surgical  instrument  connector  and  storage  rack  based  on  lessons  learned  in 
2012  USUHS  trials;  improved  robustness,  intuitive  usage,  safer  instrument  storage 
(pointing  away  from  user) 

o  Addition  of  force  sensors  to  needle  holders  and  scissors  to  distinguish  closure  on  tissue, 
needle,  suture,  etc.  and  algorithms  for  forceps,  scissors  and  needle  holders  to 
distinguish  contact  between  hard  and  soft  tissues/objects 
o  Upgrade  and  expansion  of  data  acquisition  and  global  scoring  system  to  include  video 
capture  synchronized  with  data  acquisition,  tools  for  review  and  annotation  of  data  sets, 
substantial  progress  in  gesture  recognition  and  state  machine  development 
o  Creation  of  scenario  content  and  integration  with  simulation  elements  of  data 
acquisition  architecture,  creation  of  content  frame  structure  allowing  branching  paths 
through  scenario 

o  Creation  of  augmented  reality  stereomicroscope  hardware  with  sensing  of  microscope 
position  relative  to  anatomical  structures  and  zoom  level  sensing,  creation  of  software 
libraries  to  generate  graphical  overlays  combined  with  microscope  views  of  anatomy 
o  Testing  of  Leap  Motion  and  Polhemus  sensing  systems  for  suitability  of  use  with 
simulator  system 

•  Development  of  improved  anatomical  models  and  sensing  actuation  systems  for  standalone  and 
mannequin-integrated  simulator  components 

o  Design  and  fabrication  of  eye  motion  systems  with  pitch/pan,  proptosis  and  blinking 
actuation 

o  Transition  of  anatomical  designs  from  life-cast/molded  version  towards  CT-derived 
skeletal  and  soft  tissue  structures,  inclusion  of  laceration  feature  into  CAD  models 
o  Creation  of  unified  tarsal  plate/ligament  structures  for  improved  realism,  addition  of 
vessels  and  canaliculi  into  eyelid  models,  sensing  wires  into  canthal  ligaments  for 
detection  of  cantholysis 

o  Initial  work  to  create  conjunctiva  structure  linking  eyelid  and  globe  models 

•  Integration  of  head/neck  structures  with  commercial  mannequin  system 

o  Acquired  Laerdal  SimMan  Essential  mannequin,  access  to  Laerdal  programming  SDK, 
CAD  models  of  key  components  of  mannequin  for  physical  mounting,  details  on 
electrical,  data  and  pneumatic  systems  of  mannequin 
o  Designed  &  fabricated  mounting  structure  for  neck 

o  Linked  mannequin  DC  power  supply  to  microcontroller  and  solenoid  driver  circuitry 
o  Linked  mannequin  pneumatic  supply  to  neck  motion  solenoid  valve  and  air  muscle 
systems 

•  Demonstration/data  collection  exercise  at  2013  USUHS  Ocular  Trauma  course 
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o  Extension  of  human  subjects  research  protocol 

o  Collection  of  20  data  sets  for  lid  laceration  repair  from  12  attendings,  10  residents 
o  Collection  of  survey  data  for  feedback  on  simulator,  results  showing  areas  for 
improvement  of  physical  structure,  anatomical  models  and  presentation  of  content 
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4.  Reportable  Outcomes 

Conference  papers  and  oral  presentations: 

•  Ottensmeyer  MP.  Ocular  Trauma  Training  Simulator:  Work  in  progress  -  system,  gesture 
tracking,  user  trial.  Oral  presentation,  MMVR  2013,  San  Diego,  CA,  21  Feb  2013 

•  Ottensmeyer  MP,  De  Novi  G,  Bardsley  RS,  Shah  R,  Moore  JC,  Ahn  B.  Ocular  Trauma  Training 
Simulator:  System  Description  and  Initial  User  Trials.  Syllabus  abstract,  MMVR  2013,  San  Diego, 
CA,  21  Feb  2013 

Conference  &  workshop  posters: 

•  Ottensmeyer  MP,  De  Novi  G,  Ahn  B,  Bardsley  RS,  Shah  R,  Munteanu  E,  lorino  S,  Dawson  SL. 
Development  of  an  Ocular-Facial  Trauma  Treatment  Training  Simulator.  Poster  presentation, 
MHSRS  2012,  Ft.  Lauderdale,  FL,  13-16  Aug  2012 

•  Ahn  B,  Moore  JC,  Zivieri  S,  De  Novi  G,  Bardsley  RS,  Ottensmeyer  MP.  Ocular  and  Craniofacial 
Trauma  Simulator  Development:  AR  Surgical  Microscope,  CT-Derviced  Physical  Anatomy,  Sensor 
&  Actuator  Systems.  MGH  Scientific  Advisory  Council  Poster  Session,  Boston,  MA,  20-21  March 

2013 

•  De  Novi  G,  Bardsley  RS,  Festi  M,  Ottensmeyer  MP.  Ocular  and  Craniofacial  Trauma  Simulator: 
Event  Driven  Gesture  Recognition,  Contextual  Feedback  Interface  and  User  Trial  Result.  MGH 
Scientific  Advisory  Council  Poster  Session,  Boston,  MA,  20-21  March  2013 

Invited  presentations: 

•  Ottensmeyer  MP.  Eye  trauma  simulation:  review  and  novel  system  development.  5th  Military 
Vision  Symposium  on  Ocular  and  Vision  Injury,  Boston,  MA,  18  Sept  2012 

•  Ottensmeyer  MP.  Simulation  -  grand  concepts  &  eye  trauma  example.  Roadmap  for  Simulation 
in  Eye  Care  Workshop,  Boston,  MA,  17-19  Oct  2012 

•  Ottensmeyer  MP,  De  Novi  G,  Bardsley  RS.  Ocular  and  Craniofacial  Trauma  Treatment  Training 
System:  Program  overview  &  work  in  progress.  TATRC  booth  demo  and  poster,  MMVR  2013,  San 
Diego,  CA,  21  Feb  2013 

•  Ottensmeyer  MP.  Development  of  an  Ocular  and  Craniofacial  Trauma  Treatment  Training 
System.  JPC-l/TATRC  CCTI  In-Progress  Review  presentation.  Ft.  Detrick  MD,  19  June  2013 

Conference  extended  abstracts  submitted  for  review 

•  Ottensmeyer  MP.  Development  of  an  Ocular-Facial  Trauma  Treatment  Training  Simulator. 
MHSRS  2013,  Ft.  Lauderdale,  FL,  11-15  Aug  2013 

•  De  Novi  G,  Loan  G,  Festi  M,  Ottensmeyer  MP.  Events  Driven  Surgical  Gesture  Recognition 
Validation  on  Eyelid  Laceration  Surgery  Simulation.  IMSH  2014,  San  Francisco,  CA,  25-29  January 

2014 

•  Ottensmeyer  MP,  De  Novi  G.  Conversion  of  stereo  surgical  microscope  for  augmented  reality 
application  in  an  eye  trauma  simulator.  MMVR  2014,  Manhattan  Beach,  CA,  19-22  Feb  2014 
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Graduate  thesis  manuscripts: 


•  Master's  thesis,  Stefano  lorino,  Politecnico  De  Milano,  Italy:  'TrackSim:  UN  SIMULATORE  Dl 
INTERVENTI  CHIRURGICI  PER  TRAUMI  OCULARI  E  CRANIOFACCIALI"  ("TrackSim:  A  simulator  for 
eye  trauma  and  craniofacial  surgery") 

Training  experiences: 

•  Internship:  Simone  Zivieri,  master's  candidate,  mechanical  engineering.  University  of  Modena 
and  Reggio  Emilia.  6-month  work  term  learning  mechanical  design,  CAD,  control  of 
servomechanisms,  calibrating  elastic  response  of  transmission;  working  on  the  eye  motion 
mechatronics.  Graduated  April  2013,  cum  laude. 

•  Internship:  Martina  Festi,  master's  candidate,  computer  science.  University  of  Bologna,  learning 
to  use  C#  libraries,  bridging  methods  between  C#  and  C++,  as  applied  to  interfacing  simulator  to 
Laerdal  mannequin  SDK,  assembly  language  coding  as  applied  to  simulator  data  acquisition 
microcontroller.  Graduation  expected  towards  end  of  2013. 

•  Internship:  Federico  Pifferi,  master's  candidate,  mechanical  engineering.  University  of  Modena 
and  Regio  Emilia,  6-month  work  term  in  progress,  learning  CAD,  mechanical  design,  control  of 
pneumatic  actuators,  design  of  pneumatic  actuators.  Working  on  design  of  animatronic  neck  of 
mannequin. 

•  Post-doctoral  fellowship  completed,  promoted  to  Assistant  in  Research  (MGH)/lnstructor 
(Harvard  Medical  School):  Gianluca  De  Novi,  Ph.D.  (U.  Bologna)  Robotics.  Developing  software 
architecture  and  implementation  of  simulator  interface. 

•  Post-doctoral  fellowship  completed,  accepted  employment  in  industry:  Bummo  Ahn,  Ph.D. 
(KAIST,  Korea)  Mechanical  engineering.  Developing  mechatronic  systems  for  simulator, 
instrument  motion  sensing,  intervention  sensing. 

•  Practical  training  in  manufacturing  by  laser  cutting:  Mark  P.  Ottensmeyer,  Ph.D.,  Ryan  Bardsley. 

Funding  applications 

•  Proposal  submitted:  Open  Architecture  Virtual  Autopsy/Anatomy  Table  (VAT),  Gianluca  de  Novi, 
Ph.D.,  P.l.  CIMIT  Boston  Simulation  Consortium  Innovation  Award  competition.  Based  on 
gesture  recognition  concept  and  graphical  rendering  techniques  developed  for  and  in  parallel 
with  craniofacial  simulator 

•  Proposal  submitted:  Eye/Face  Trauma  Simulation  Enhancements,  Mark  P.  Ottensmeyer,  Ph.D., 
P.l.  Subaward  proposal  for  CIMIT  application  to  Joint  Warfighter  Medical  Research  Program, 
extending  work  previously  supported  by  CIMIT.  Proposal  to  develop  mid-facial  fracture  module, 
adapt  simulator  to  be  compatible  with  CIMIT  modular  mannequin  system  under  development. 

•  Proposal  in  preparation:  Advanced  Modular  Manikin  solicitation  W81XWH-13-R-0032.  James  A. 
Gordon,  M.D.,  P.l.  Program  to  develop  platform  technology  for  a  modular  manikin  systems, 
adapting  ocular-craniofacial  simulator  for  use  as  one  possible  head  module  for  this  system. 

Invention  disclosures  and  patent  applications 
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•  Invention  disclosure  MGH  22084:  Includes  simulator  technological  elements: 

o  Augmented  Reality  Stereo  Surgical  Microscope 

o  Animatronic  eye  motion  mechanism  for  eye  trauma  simulator  with 
pitch/yaw/blink/proptosis  motions,  replaceable  eye  lid  and  eye  globe  modules, 
treatment  sensing  and  physiological  response 
o  Sensors  for  detection  of  incision  creation  and  incision  path  for  medical  simulation 

•  Invention  disclosure  MGH  22275:  "Training  Simulator  for  Ocular  Trauma  and  Other  Domains 
Using  Event-based  Surgical  Gesture  Detection  and  Multiple  Instrument  and  Hand  Motion 
Tracking  Systems",  which  described  additional  content  presentation  and  use  of  reflected  iR 
imaging  using  a  Kinect-type  device  for  hand  tracking 

•  United  States  Provisional  Patent  Application,  filed  02/20/2013,  Serial  Number  61/766,987, 
covering  MGH  22084 

5.  Conclusion 

This  report  presents  the  progress  made  in  the  second  year  of  our  program  to  create  a  novel  eye  and  face 
trauma  simulator  for  use  by  trainee  surgeons  and  those  new  or  returning  to  eye  trauma  practice  as  well 
as  first  responders  facing  eye  trauma  and  likely  coincident  facial  trauma  in  the  field. 

Based  on  lessons  learned  and  user  testing/demonstrations  of  Year  1,  we  created  what  is  essentially  the 
completed  version  of  the  standalone  simulator  platform  for  surgeons  and  surgical  trainees.  As  a  result 
of  the  first  year's  demonstration,  we  were  invited  to  return  to  the  2013  USUHS  Ocular  Trauma  course  to 
serve  as  a  demonstration  of  simulation-based  training  for  eye  trauma  in  parallel  with  our  own  testing 
and  data  collection  needs.  At  this  event,  we  were  able  to  include  improved  content  that  was 
synchronized  with  the  sequence  of  actions  of  the  trainees.  Based  on  this  event,  an  invitation  to 
participate  again  in  2014  was  issued  by  the  USUHS  course  director.  Only  minor  alterations  are  expected 
to  be  made  on  the  system  hardware,  and  advances  in  the  simulation,  data  collection  and  analysis 
software  are  expected  to  make  the  system  more  suitable  as  a  teaching  device  (and  potential  commercial 
product). 

Substantial  work  was  performed  towards  the  final  goal  of  adapting  the  head  and  neck  of  the  mannequin 
for  use  as  a  replacement  for  a  commercial  mannequin's  head.  We  acquired  a  Laerdal  SimMan  Essential 
mannequin  as  a  platform  for  modification,  began  the  process  of  creating  software  links  between  our 
architecture  and  the  mannequin's  controls,  developed  prototype  systems  for  integrating  our  actuators 
and  sensors  with  the  power  and  data  systems  of  the  mannequin,  and  fabricated  a  mounting  structure  to 
attach  the  work-in-progress  neck  to  the  mannequin's  internal  frame. 

Improvements  to  the  anatomical  realism  of  the  system,  from  the  ligaments  and  tissues  of  the  eyes  to 
animatronic  systems  to  animate  the  eyes  to  the  inclusion  of  vessels  and  ducts  in  the  eyelids  have  been 
accomplished.  These  will  serve  as  the  basis  for  the  remaining  eye  trauma  modules  to  be  completed 
during  Year  3. 
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The  work  has  been  presented  to  multiple  experts,  both  our  own  SMEs  and  unrelated  experts  in  eye 
surgery,  and  response  has  been  positive.  We  anticipate  that  the  future  modules  and  improvements  will 
continue  to  meet  with  approval.  For  the  eye  trauma  elements  contacts  with  some  of  these  experts  is 
likely  to  lead  to  additional  opportunities  for  collecting  user  performance  data  to  help  develop  the 
scoring  and  feedback  system. 


Implications  of  successful  completion  of  this  program  lie  in  a  number  of  directions. 

The  first  is  that  a  useful  eye  trauma  training  system  will  provide  a  new  tool  for  teaching  basic  skills  to 
young  surgeons  with  limited  trauma  exposure.  Such  a  tool  will  be  versatile,  in  that  the  base  platform 
will  support  multiple  trauma  scenarios,  initially  including  simple  and  complex  lid  lacerations,  lacerations 
and  punctures  of  the  anterior  hemisphere  of  the  globe,  and  retrobulbar  hemorrhage.  These  will  be 
more  controllable,  repeatable  and  predictable  than  the  use  of  cadaver  tissue  or  live  animal  materials  for 
training. 

The  elements  of  the  system  that  address  the  first  responder  will  allow  more  comprehensive  training 
scenarios  beyond  those  available  through  current  commercial  mannequins,  in  which  head  and  face 
trauma  are  represented  typically  by  non-active,  rubber  masks  or  moulage  placed  on  an  otherwise 
healthy  mannequin  head/face.  Realistic  and  responsive  eye/face  trauma  will  improve  training  by 
helping  trainees  to  recognize  that  trauma  is  not  limited  to  the  extremities  or  torso  of  the  casualty  and 
remember  to  check  for  it  in  the  field. 

The  tracking  system  and  scoring  methods  that  we  are  developing  have  application  beyond  eye  trauma. 
The  great  majority  of  the  research  into  measuring  surgical  performance  produces  global  scores  that  may 
distinguish  between  expert  and  novice  performance,  but  cannot  guide  improvement  at  the  sub-task  or 
gestural  level  -  informing  a  trainee  that  they  must  increase  overall  speed  or  make  their  motions  smaller 
is  of  limited  utility,  as  it  does  not  specify  which  elements  of  their  performance  is  too  slow  and  which  are 
of  acceptable  speed,  or  where  they  need  to  make  their  motions  more  compact  and  efficient.  We 
believe  that  the  event-based  gesture  detection  and  tracking  system  will  be  suitable  for  quickly 
identifying  where  a  trainee  diverges  in  performance  from  that  of  an  expert,  and  given  a  well-designed 
suite  of  tips  and  instructions  based  on  experts'  experience,  can  provide  timely  guidance  to  accelerate 
learning.  This  applies  equally  well  to  eye  trauma  as  it  would  to  many  other  types  of  surgery  or  tasks 
requiring  manual  dexterity  together  with  cognitive  skills  required  for  a  complex,  multi-step  process. 
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